A public spending watchdog has demanded action to speed up the Government's "frustratingly poor" record on removing from the UK foreign criminals who are costing taxpayers hundreds of millions of pounds.
Too many overseas inmates are still being locked up at public expense as the rate they are sent home has dropped by 14% over the past four years, the Commons Public Accounts Committee (PAC) said.
MPs also called for improvements in under-performing new prisons and questioned the closure of several which had been among the best in the country as part of a successful money-saving programme.
In December, the National Audit Office said speeding up the removal of foreign prisoners - who number about 11,000 and make up 13% of the prison population - was the best way to reduce the bill for jails,
The PAC said the National Offender Management Service (Noms) " should work with the Home Office to better understand the reasons for delays in removing foreign national offenders, tackle the barriers to removal, and take all steps to improve performance in this area".
Its report welcomed progress in securing a significant improvement in value for money, with new prisons providing good, modern accommodation.
But inmates still routinely share cells, some in overcrowded conditions. And in a bid to make savings, some high-performing prisons were closed before newly built ones started to perform well, it noted.
More space could be freed up by giving prisoners serving indeterminate sentences more access to courses that would reduce their risk to the public, allowing the Parole Board to release them, it suggested.
The PAC said the poor performance of new prisons HMP Oakwood and HMP Thameside - which were among only three give the lowest quality rating last year - should be addressed "as a matter of urgency".
Any failure to get them up to four-star standard by 2014/15 would require an explanation to the committee, it said, including improvements in the amount of " good-quality purposeful activity" being provided.
Prisons' individual performance should be considered when deciding which to close, the MPs suggested, in addition to other factors.
The committee's chair, Labour MP Margaret Hodge, said: "Performance in reducing the number of foreign national prisoners continues to be frustratingly poor, costing the taxpayer hundreds of millions of pounds.
"While more than 1,000 foreign national offenders are deported each quarter, a similar number are convicted, so the overall number of foreign national prisoners stays at the same level of around 11,000 - 13% of the total prison population.
"The agency should work with the Home Office to understand why there are delays in removing foreign national offenders, and tackle the barriers to their removal."
She welcomed the " significant savings in running costs" achieved by Noms, which is on track to achieve reductions of £70 million a year and a good standard of accommodation on time and within budget.
"The programme has been well managed and has benefited from experienced and consistent leadership," she acknowledged .
"The strategy has not, however, done as well at creating constructive regimes in which to work with offenders to support rehabilitation and prevent reoffending by preparing prisoners for work.
"Although the two new large, contracted-out prisons were constructed on time, they have not performed well since opening.
"Neither prison gives enough priority to meeting offenders' rehabilitation needs. HMP Oakwood does not provide enough education time, and relies too much on less purposeful activities such as cleaning. HMP Thameside has no workshops.
"On the other hand, the agency has closed some prisons that were performing well. Although it considered a range of criteria in deciding which prisons to close, such as their geographic location and relative running costs, the agency did not take their performance into account.
"Three of those closed had recently been awarded top performance ratings."
There was "much room for better performance in preparing prisoners for release at the earliest opportunity", she suggested.
"In June 2013, the prison population included more than 6,000 prisoners with indeterminate sentences who had served the minimum term of their sentence so potentially could be released.
"However to qualify, many prisoners are expected to attend courses like behaviour management; yet these have been cut, making it less likely for many prisoners to secure release."
Justice Minister Jeremy Wright said: "The foreign national prisoner population is lower than it was in 2010 and reducing it further is a top priority for this Government. We are working hard to reduce the numbers in our prison system - in 2012 alone we deported more than 4,500 foreign criminals from the UK.
"Under this Government we have made it a priority to establish compulsory prisoner transfer agreements. We have also increased the numbers removed through the Early Removal Scheme and the introduction of the Tariff Expired Removal Scheme. We are also working closely with the Home Office to ensure removal processes are as efficient as possible."