Colombia is on the verge of ending half a century of conflict, but bereaved locals in ruined regions such as Cauca struggle to believe that peace will come easily.

In the mountain town of Toribio, bullet holes in the walls show the violence that has ravaged this region, in fighting between state forces and guerrillas.

"Between 1982 and 2015 there were 6,000 armed clashes. They were our daily bread," said local leader Mauricio Casso, a member of the Nasa indigenous group.

"There are thousands of widows, orphans and people maimed, and hundreds dead. The park used to be deserted. The streets were closed off. The children did not play outside," he told AFP.

At peace talks hosted by Cuba, the Colombian government and the leftist FARC rebel force are closing in on a peace deal to end the conflict.

They had hoped to sign a deal this Wednesday, but that deadline now looks like it will pass with key issues in the negotiations still unresolved.

Progress in the talks has eased the atmosphere in Cauca since the FARC declared a ceasefire in July, but security forces remain on alert.

The local police station lies behind high green walls pocked with bullet holes, its entrance protected by a bunker of sandbags.

"The mountains are nearby. The guerrillas used to fire from there. They are still around there. But we haven't heard any shooting for more than a year," said Major Rodriguez, head of the local brigade.

"I hope it lasts."

The road in front of the police station is still torn up from where attackers blew up a bus there, wounding 70 people and killing a policeman. The blast was blamed on the FARC.

Nearby mud houses damaged by the explosion are now covered in paintings calling for peace.

- Families driven from homes -

The Colombian conflict started as an uprising by the Marxist FARC in 1964 and ground into an entrenched territorial conflict, drawing in various rebel groups and drug gangs.

It has killed 260,000 people and left 45,000 missing and 6.6 million displaced, according to officials and NGOs.

The conflict has driven families from their homes.

Mariela Patino, 56, is still looking for her son, who she says was abducted by guerrillas in 2003. Threats drove her to abandon the farm where she lived and move to another town.

"I reported it and started putting up notices everywhere. The FARC told me if I didn't leave they wouldn't be responsible for my life or my family's."

- Left to grow drugs -

Cauca's mountainous terrain also draws the drug trade which has fuelled the conflict. Its winding roads are lined by coca and marijuana plantations.

Among the local people sorting marijuana leaves by the roadside is Fernanda, 21. She gets paid about 90 dollars for helping pick a crop of the herb, which is harvested four times a year.

"If the state helped us, we would plant something else," she says. "But here we get nothing. We are abandoned."

Locals fear that if the rebels and the government make peace, the drug gangs will move in -- or other groups such as the left-wing ELN, a rival rebel force not party to the peace talks.

"Here in the north of Cauca, in recent months we have been seeing a big presence of ELN. They have painted on the walls and held rallies," said Giovani Yule Zape, representative of a grouping of indigenous groups in northern Cauca.

On Tuesday the Office of the UN High Commissioner for Human Rights hailed progress towards peace in Colombia. But it warned the deal risked leaving a power vacuum that other armed groups would fill, fighting over drugs and other sources of criminal revenue.

It said the breaking up of various armed groups posed a "permanent challenge to peace."

Local woman Yolanda Millan, 58, is shocked to see pro-ELN graffiti stenciled on a hut in her village of El Palo.

"That's new," she says. "Who knows what is going to happen?"

Elsewhere, in central Colombia where the FARC presence is still strongly felt, locals in the village of La Macarena share the apprehension.

"There will never be peace," said Jairo Garcia, a butcher of 66. "The army left, the guerrillas came. The army came back. Now other groups will come to bother us."