BEIJING (CNN) -- Dozens of Chinese workers angry over a pay dispute have held their U.S. boss hostage for five days, the American businessman said.

Chip Starnes, co-founder and president of Speciality Medical Supplies China, has been trapped in the company's suburban Beijing factory since Friday. He told CNN it's all because of a misunderstanding about layoffs and severance packages.

"I tried to leave a day and a half ago, and there was like 60 or 70 of them here inside every entrance, and every exit was barricaded," Starnes said Tuesday from behind the gates of the factory. "I can't go anywhere."

Starnes was calm but looked haggard, wearing the same blue jeans and a blue shirt that he said he has been wearing since last week.

The incident started when the company laid off about 30 workers in its injection molding division. He said he transferred workers to another division, but some workers did not want to move. So the company gave them severance packages.

Starnes said the problem came when workers who already moved to another division also wanted severance packages.

But workers who spoke to CNN from behind the factory gates gave an entirely different account. Some said they're owed two months' worth of back pay and feared everyone at the plant would be laid off, as they claimed no new materials had entered the factory.

Starnes disputed that.

"There are no outstanding balances. There is no reason to be doing this other than bad information," he said. "The wrong information got out. Things snowballed. I am in the process of trying to fix that, but the demands and the amount of money that it is taking to do that puts the company in a bad situation, so there has got to be some compromise."

About four police officers at the scene would not comment on the situation.

U.S. embassy officials in Beijing met with Starnes on Monday and confirmed he is safe and is close to reaching an agreement, spokesman Nolan Barkhouse said. He said the executive has access to attorneys.

Starnes said the employees haven't caused any physical harm but have disrupted his sleep by banging on doors and windows.

He has a cot and is receiving three meals a day, said Les Capella, the U.S president of Speciality Medical Supplies.

Capella said the company has offered protesting employees a lesser severance package if they want to leave. But he said the 80 to 100 employees at odds are demanding about $500,000 -- which would bankrupt the company.

CNN's John Zarrella contributed to this report.