A number of those supporting a protest outside the Combat Stress centre at Audley Court, Newport, have expressed surprise at the comments from the charity's chief executive, Sue Freeth, earlier this week.
In a letter Ms Freeth said that staff had been asked to work from home for two weeks because of abuse and verbal threats to staff at the centre.
A number of former members of the armed forces have joined former Gus Hales outside the centre more than a month, in protest at the care veterans have received from the charity.
Mr Hales, 62, began the protest with a hunger strike, and received an unconditional apology from the charity of the way it handled his discharge from its care.
He has called off the hunger strike after becoming seriously ill but has since continued the protest, with a renewed call for an independent inquiry into the care of people treated by Combat Stress.
Ben Goodchild, 50, of Saddleworth Moor, said he had seen no harassment of workers.
The former member of the Light Infantry, who did his basic training in Shrewsbury, said: "No one has been threatened, it has been a fairly laid back atmosphere."
Ms Freeth's letter said that veterans receiving treatment at the centre had also asked for sessions to be moved to another location.
Mr Goodchild said that he and his fellow veterans would not want to do anything that would harm other former servicemen seeking treatment.
He said: "At one point in my career I would have died for any one of these lads here. Why would I stand there and abuse the people that I want to help? None of us here would do it."
He added: "We do not want to be made to look like animals when we are not. We are trying to win hearts and minds, we are not stupid, we are not threatening people."
Tony Smith, 58, a former member of the Staffordshire Regiment, who served in Northern Ireland, said the veterans had been upset by the claims.
He said: "Given the fact that it is a peaceful demonstration we have done everything by the book. I have not seen any instances of the staff being intimidated.
"We have insisted all along that this is a peaceful demonstration, it is not about intimidating people who go there or work there.
"The presence needs to be outside but we have been very specific and caring about people who are going to work there."
Norman McGuigan, 57-year-old former Light Infantryman, who trained in Shrewsbury, said he had been "disappointed" at the comments.
He said: "That has not been happening at all, we have done everything respectfully from day one."
Mr McGuigan did accept that protesters had attempted to film through the windows of the charity, an issue highlighted by Ms Freeth.
He said: "We have filmed through the windows, we won't deny that. But it was purely to find out what was going on."