President Salva Kiir and rebel boss Riek Machar inked a deal "ending the war," said head mediator Seyoum Mesfin of the East African regional bloc IGAD.
The two rivals, who first shook hands and then prayed together, agreed to end all fighting within 24 hours of signing the document.
The United States, a key backer of South Sudan's push for independence from Khartoum, has poured in billions of dollars in aid to the country since it split from Sudan in 2011. It lobbied intensely for the deal.
"Today's agreement to immediately stop the fighting in South Sudan and to negotiate a transitional government could mark a breakthrough for the future of South Sudan," US Secretary of State John Kerry said in a statement.
"The hard journey on a long road begins now and the work must continue," he added.
"We urge both leaders to take immediate action now to ensure that this agreement is implemented in full and that armed groups on both sides adhere to its terms."
Kerry had brandished the threat of targeted sanctions against Kiir and Machar and Washington slapped sanctions on senior military commanders.
"I saw with my own eyes last week the stakes and the struggles in a new nation we helped courageous people create," he said, referring to his recent African tour in which he pushed for the two leaders to sit down for talks.
"The people of South Sudan have suffered too much for far too long."
National Security Adviser Susan Rice, a former assistant secretary of state for African affairs, agreed that "fighting in South Sudan has robbed that country of hope and denied its people the peace and prosperity they deserve."
While both South Sudanese leaders promised peace, fierce fighting still rages, amid United Nations warnings of the risk of severe famine and genocide.
The war has claimed thousands -- and possibly tens of thousands -- of lives, with over 1.2 million people forced to flee their homes.
Aid agencies are warning that South Sudan is now on the brink of Africa's worst famine since the 1980s.
Kerry said the agreement "presents an opportunity to start a path towards peace that must not be lost. We will do all we can to help."
The deal "holds the promise of bringing the crisis to an end," Rice added.
"We urge President Kiir and Mr Machar to move swiftly to honor the agreement in word and deed by ending the violence and negotiating in good faith to reach a political agreement that can ensure stability, prosperity and peace for all of South Sudan's people."