A survey of 3,500 workers by the GMB union found that four in five voiced concerns about going back to work as the lockdown eases.
Almost two in three said they were worried about being pressured into returning to work, and most described advice as “unclear”.
The GMB said it discovered “disturbing” comments from workers being bullied into returning, while some voiced fears about their safety.
Comments included: “The manager where I work – who has not set foot on the premises since the outbreak of the pandemic – recently sent an email instructing staff not to use PPE (masks) unless service users are infected.
“Management need to come back to the yard instead of barking orders from the safety of their homes.”
John Phillips, acting general secretary of the GMB said: “We need a safe and managed return to normal working, but that isn’t happening yet.
“Those facing going back to work need reassurance their welfare and those of their families is being put first, but too many are living in fear of the danger they and their loved ones are now facing.
“The Government’s unclear advice and failure to enforce workplace standards means risky behaviour by some employers could see all of us punished with a resurgence of the virus that could destabilise our country’s recovery.
“It’s on ministers to get this right. They need to be led by the evidence and provide clear plans for the safe return to work.
“Their responses to this crisis would be an international laughing-stock if the consequences of their failures were not so deadly serious.”
A Government spokesman said: “We know this is an unsettling time for workers. Employers should make every effort to support working from home, including by providing suitable IT and equipment.
“For those that cannot work from home we have published clear and comprehensive guidance to give businesses and workers the confidence to return to work safely, led by the best available science.
“We urge employers to listen to the concerns of their workforce and take socially responsible decisions. If they do not, workers can report them to their local authority or the Health and Safety Executive who can take a range of action.”