The pilgrims were crossing the Kamala River, which flows through Nepal and northernIndia, after early morning prayers at a nearby temple, Manohar Jha, a district police official, told AFP.
"They successfully crossed the river on their way to the temple. But the river was flooded when they returned at around 6am local time," Jha said.
A seventh pilgrim remains missing, Jha said.
Incessant rain since Sunday night has left hundreds of Indian and Nepali pilgrims stranded at the temple, he added.
It was still unclear whether the drowned pilgrims were Nepali or Indian.
Thousands of pilgrims visit the Mahadev temple during the month of Sawan (July 15 to August 15).
This year's monsoon rains have devastated rural communities in mountainous Nepal, destroying crops and damaging thousands of homes.
According to government figures, 60 people were killed in floods triggered by heavy downpours in June and July.
Hundreds of people die every year from flooding and landslides during the monsoon season in Nepal.