23 July 2011 Last updated at 08:23 ET
Col Gai was shot in the early hours of Saturday morning despite having apparently been reconciled with the South Sudanese army.
South Sudan became independent earlier this month, but is facing at least half a dozen armed rebellions.
The incident happened at or near the district of Pakur in Unity State.
Colonel Gai began his rebellion following elections last year, after the candidate he supported for state governor was controversially declared to be the loser.
This week Col Gai signed a peace agreement with the South Sudan army and was due to be reintegrated into it.
 |
| Rebels accuse the South Sudanese army of being responsible for luring Col Gai to his death |
The army spokesman denied this.
He said Col Gai had been killed in a fight with his deputy, after he had changed his mind about the peace agreement.
Ethnic divide
But it is clear this incident will make the remaining rebel leaders less likely to accept the amnesty offered by President Salva Kiir.
According to a rebel leader who knew him, Gatluak Gai was in his 50s, with an imposing, physical presence.
Although he was not an educated man, he was able to recruit soldiers from the sub-sect of his Nuer ethnic group, which has often been in conflict with South Sudan's biggest group, the Dinka.
Col Gai is survived by many children, and nine of his boys fought in his militia.
One of his daughters is married to Gen Peter Gadet, who leads the most active rebel group in South Sudan.
bbc.co.uk/news/world-africa-14261525