Senior Labour party figures have described the comments as “appalling”.
Shadow home secretary Yvette Cooper said spiking – putting alcohol or drugs in someone’s drink or body without their consent – was a “disturbing and serious crime which is having a devastating impact on young women’s lives”.
“It is truly unbelievable that the home secretary made such appalling jokes on the very same day the government announced a new policy on spiking,” she added.
And charity Women’s Aid said political leaders were relied upon to “take action to end violence against women and girls, and the misogyny that underpins it”.
“It is vital that spiking survivors see ministers treating the subject seriously and not downplaying the reality so many women face,” it said on X.
Another women’s rights organisation, the Fawcett Society, called on Mr Cleverly to resign, asking: “How can we trust him to seriously address violence against women and girls?”
In a statement, it said: “It’s sickening that the senior minister in charge of keeping women safe thinks that something as terrifying as drugging women is a laughing matter.”
A spokesman for the home secretary said: “In what was always understood as a private conversation, James, the home secretary tackling spiking, made what was clearly meant to be an ironic joke – for which he apologises.”
A source told the BBC he did not recollect the exact wording he had used, because it was a private off-the-record event, but recognised that any joke along those lines was inappropriate – which was why he was apologising.
The incident happened on 18 December, when political journalists were invited to a drinks reception in 10 Downing Street along with political aides, ministers and Prime Minister Rishi Sunak.
On the same day, Mr Cleverly had promoted a raft of new government measures to tackle spiking and described it as a “perverse crime”.
According to reports first published by the Sunday Mirror newspaper, Mr Cleverly told some female guests there that “a little bit of Rohypnol in her drink every night” was “not really illegal if it’s only a little bit”.
It also says he laughed that the secret to a long marriage was ensuring your spouse was “someone who is always mildly sedated so she can never realise there are better men out there”.
Conversations at Downing Street receptions are usually considered “off the record” – and therefore not reported on – but the Sunday Mirror said it had decided to break this convention because of Mr Cleverly’s position and the content of his remarks.
Mr Cleverly met his wife at university and the couple have two children.
Labour’s shadow minister for domestic violence and safeguarding, Alex Davies-Jones, said: “If the home secretary is serious about tackling spiking, and violence against women and girls, then that requires a full cultural change.
“The ‘banter’ needs to stop and it has to start at the top.”