Question about Husband's behaviour, please answer!?

We have been together for past 4 years and we have two kids (2 yrs and 9 months). There is something which is causing problems in our life. My husband wants to have a two weeks holiday by himself once a year. Now he wants to go to Bangkok for a week. Also note that he works in Africa and we live in Europe and he comes to see us every two months for two weeks. I find it difficult to accept. He thinks I am possessive and am behaving in a strange way by not supporting him in his pion for travelling. We have been fighting over this and now I am not able to figure out what is the right thing to do for me. Please advise!