Amnesty International said that Libya risked repeating the violations that led to the uprising that removed the dictator after 42 years in power last August.
On Saturday, Libyans will chose a congress to appoint a cabinet and a body to draft a new constitution.
But the transitional authorities, Amnesty said, have not only failed to control militias, but effectively granted them immunity from prosecution in the name of protecting the revolution.
The human rights group's 72-page report documents dozens of cases of arbitrary arrest, long-term detention, beatings, abuse and torture of anyone suspected of loyalty to Gaddafi.
It visited 15 unofficial prisons and spoke to a former government employee, who was arrested with her sister in the east of the country in February.
She was taken to a farm where she was suspended from a door for hours, had boiling water poured over her head and was beaten and stabbed.
Her sister was also badly beaten while the husband has disappeared. She was released without charge.
"It is deeply depressing that the authorities have failed so comprehensively to break the grip of the militias on Libyan security," said Hassiba Hadj Sahraoui, Amnesty's deputy director for northern Africa.
Militias vary in size from gangs guarding neighbourhoods to small de facto armies of the sort in Zintan that is holding Gaddafi's son Saif al-Islam and which detained an International Criminal Court lawyer and her translator for a month after they visited him.
They often take their names from comrades killed in the conflict against the late dictator and boast that in the absence of a strong central authority they alone can guarantee security.
Operating above the law, militia raids on suspects often involve theft of cash, gold, cars and anything they can lay their hands on, said Diana Eltahawy, the author of the report.
Most are refusing to disarm or join the national army or police force. The ministry of interior told Amnesty that it had dismantled just four militias in Tripoli.
"What needs to change is that the government needs to at least admit what is happening. They refuse to admit how widespread these practices are. If they don't control it, it will be a very, very difficult culture to change," she said.
Isn't the new, democratic Lybia a great place? Isn't it wonderful that the evil dictator is deposed?
Not to worry though, as soon as the elections occur armed groups will be rushing to lay down their weapons, right? Right?