The number of reported cases of abuse against vulnerable adults rose by 23% in just one year, provisional figures show.
Data from 99 councils in England show that the number of reported cases rose by 21,000 between 2010/11 and 2011/12.
There were a total of 130,000 cases of abuse reported to 121 councils in the same period. Of these, 108,000 cases were investigated by local authorities, according to provisional figures from the Health and Social Care Information Centre (HSCIC).
Care workers and family were the most common alleged abusers, according to the report. Councils received 31,000 allegations about social care workers and 24,000 about family members.
Three in five of the reports were about abuse of women and almost half were concerning disabled people, the HSCIC report found. And 60% were concerning people aged 65 and over. The rate of reported incidents was highest in the Midlands and lowest in the South West.
Physical abuse was the most common type of abuse, accounting for 29% of all allegations, this was followed by neglect which was attributable for 26% of reported incidents.
Vulnerable adults, described in the report as people who are or may be in need of community care services because they are elderly or suffer mental illness, a disability or another illness, were more likely to be abused in their own home - accounting for 40% of all locations noted in the report. But more than a third of cases (36%) occurred in care homes.
Of the 83,500 incidents investigated, 41% were either substantiated or partially substantiated, figures show.
A Department of Health spokeswoman said: "We should consider it a scandal that not everyone is treated with the respect and dignity they deserve. Where evidence or allegations of abuse are uncovered, swift and sure action should be taken by regulators or the police.
"We are committed to uncovering abuse wherever it takes place and this data is an important part in shining a light on the issue. With the Care Quality Commission undertaking hundreds of unannounced inspections, mandatory boards involving the police responding to abuse and plans for tougher board-level accountability for care failings we are taking action to bring an end to poor quality care."