The one-day visit is Sharif's first since he took office in May and comes as Karzai is locked in a public dispute with Washington over a crucial security deal covering the role of US soldiers who remain in Afghanistan after 2014.
Pakistan is seen as crucial to peace in neighbouring Afghanistan as it was a key backer of the hardline 1996-2001 Taliban regime in Kabul and is believed to shelter some of the movement's leaders.
"Mr. Nawaz Sharif has arrived in Kabul. He is going to meet President Karzai," a palace official told AFP. "They will talk about improving mutual relations between the two neighbouring countries."
Sharif will also meet the Afghan High Peace Council, which is seeking to open negotiations with the Taliban insurgents who have fought US-led NATO and Afghan forces since 2001.
Pakistan said it recently released former Taliban number two Mullah Baradar, who is seen by Kabul as important to bringing the militants to the negotiating table.
But militant sources have complained that Baradar is effectively still behind bars in Pakistan, and there has been no confirmation that the High Peace Council was able to meet with him during its recent visit to Pakistan.
Karzai, who is due to step down next year, has been stalling over the security pact with the United States that would allow some US troops to stay in Afghanistan for training and counter-terror missions after the NATO combat mission ends.
Washington is keen to complete the deal, but accuses Karzai of introducing new, last-minute conditions despite a "loya jirga" assembly that he convened voting for him to sign the agreement promptly.
The Afghan president has been bitterly critical of the United States, and on Friday rounded on NATO forces over an air strike that he said killed a two-year-old boy in the southern province of Helmand.
"This attack shows American forces are not respecting Afghan lives," Karzai said.
NATO apologised for the civilian casualties caused by the air strike, which it said targeted a Taliban commander involved in operations against Afghan security forces in Helmand.
Afghanistan's peace process has been at a standstill since a Taliban office opened in Qatar in June, enraging Karzai as it was styled as an embassy for a government-in-exile.
"Both the leaders will discuss the peace and reconciliation process in Afghanistan," Aizaz Ahmad Chaudhry, a Pakistani foreign ministry spokesman, said ahead of the visit.
The Taliban have refused to have direct contact with Karzai or with the High Peace Council, dismissing them as puppets of Washington.
Karzai and Sharif met British Prime Minister David Cameron in London in October in the fourth of a series of trilateral meetings designed to foster stability in the volatile South Asia region.