WASHINGTON (CNNMoney) -- Budget cuts at the Internal Revenue Service are causing long lines for the poor and elderly seeking free tax preparation advice, lengthier hold times for IRS help line calls and delays in tax refunds for identify theft victims.

With a hiring freeze preventing the IRS from replacing workers who retire or leave for other jobs, existing staffers say they're feeling the pressure of lack of manpower, according to the National Treasury Employees Union, which represents federal employees at the tax collection agency.

The IRS has lost 10,000 employees to attrition in just the past two years, or over 9% of its total workforce.

"We are doing as best we can with a staff of 14 that used to be a staff of 32," said one IRS employee about working conditions in a survey conducted by the union.

In an effort to prevent additional budget cuts from wreaking havoc during the height of the tax filing season, the IRS pushed back employee furloughs until the summer, after the April 15 deadline for filing tax returns.

Forced cuts have already begun to take root at dozens of other federal agencies, triggered on March 1 by $85 billion in automatic government spending cuts. Tens of thousands of defense workers have lost jobs and federal public defenders have already started to take unpaid time off at home.

The IRS has been trimming costs in preparation for the forced budget cuts, just like other agencies. That's meant that staff levels have dwindled at the 389 Taxpayer Assistance Centers nationwide. Typically, these centers offer tax and retirement counseling for the elderly and help the poor prepare their returns for free.

The National Taxpayer Advocate estimated that these centers will serve about 5.4 million people by this year's tax filing deadline, down from 6.4 million in 2011, according to a report.

IRS workers told the union that taxpayers are lining up outside the centers "beginning in the pre-dawn hours" before the assistance centers open, according to the union survey.

Budget cuts and lack of staff have increased wait times for those seeking IRS help by phone to 18 minutes on average this tax season. That's up from 13 minutes in 2011, according to the National Taxpayer Advocate's report.

The likelihood that a caller would get a live IRS employee on the phone for tax questions also fell, according to the report. This year only 63% of taxpayers who call the IRS are expected to successfully get help, down from 70% of callers in 2011. More taxpayers just hang up in frustration, the report found.

Identity theft victims are among the worst hit -- having to wait six months to a year if they're seeking tax refunds, the union said.

The situation could only get worse. Currently, IRS employees are facing between five and seven days of unpaid time off, with the first furlough possibly kicking in around Memorial Day, May 27.

The agency has also been cutting costs for travel, training, facilities and supplies, according to a February memo from IRS acting Commissioner Steven Miller.

The survey was produced from an analysis of thousands of written responses by IRS employees who wrote essays explaining how budget cuts were affecting taxpayer services, said union spokeswoman Dina Long.

A request for comment from Treasury Department was not returned.