Kharjo was hired by the merchants Ahkati and Zaynabi to protect their caravan in case of the potential attack from bandits. This is obvious due to the fact he is wearing heavy steel-plated armor, so that he can prove durable warrior in the event of these incidents.
Unlike most individuals, Khajiit caravans such as that of Kharjo's are nomadic, and thus will be found traveling nonstop at any given time, only stopping for the opening of the caravan's trading post for a short time. Because of this, Kharjo can be a difficult individual to recruit as a follower.
When asked for the first time by the Dovahkiin, he will tell them that he had his personal talisman called a Moon Amulet stolen by a recent bandit raid on the caravan, if the Dragonborn decides to retrieve the amulet back from the bandits, Kharjo tells him/her that he will be in their debt. Once the amulet is returned, Kharjo will thank the Dragonborn with a small amount of gold and the availability of recruiting him as a follower. It should be noted that as a follower he is quite adaptive to the environment, but will refuse to enter cities, as it goes against his beliefs. He can not become a potential candidate for marriage, nor can he become a steward to the Dragonborn; however he can be inducted into the Blades, making him easier to find as a follower than beforehand.