The unmarried couple said the child had been born two months ago but had not yet received a birth certificate, local media reported.
They had previously said they refused to provide DNA test results for birth registration, citing privacy and religion as reasons.
On Tuesday Secretary for Security Chris Tang denied it was a privacy issue and called on the couple to provide evidence that they are the boy's biological parents.
"At this point, the mother has not been able to provide any records of her prenatal checkups or any medical consultations," he told reporters.
She had also refused to provide photos taken while she was pregnant, he added.
"With the baby's welfare as a starting point, we will first send the baby to the hospital for a medical exam," Tang said.
The couple have also said that Swedish authorities have taken custody of their older daughter during an illegal stay in Sweden in 2023.
Their Facebook page campaigning for her return drew attention to the current Hong Kong case.
Newborns must be registered within 42 days after birth under Hong Kong law.
Anyone found guilty of child neglect faces up to 10 years in prison.
cla/reb/ami