The students were shot dead as they slept in their dormitory at the College of Agriculture in Yobe state.
North-eastern Nigeria is under a state of emergency amid an Islamist insurgency by the Boko Haram group.
Boko Haram is fighting to overthrow Nigeria's government to create an Islamic state, and has launched a number of attacks on schools.
Casualty figures from the latest attack vary, but a local politician told the BBC that around 50 students had been killed.
The politician said two vanloads of bodies had been taken to a hospital in Yobe's state capital, Damaturu.
A witness quoted by Reuters news agency counted 40 bodies at the hospital, mostly those of young men believed to be students.
College provost Molima Idi Mato, speaking to Associated Press, also said the number of dead could be as high as 50, adding that security forces were still recovering the bodies and that about 1,000 students had fled the campus.
A Nigerian military source told AP that soldiers had collected 42 bodies.
The gunmen also set fire to classrooms, a military spokesman in Yobe state, Lazarus Eli, told Agence France-Presse.
The college is in the rural Gujba district.
In May, President Goodluck Jonathan ordered an operation against Boko Haram, and a state of emergency was declared for the north-east on 14 May.
Many of the Islamist militants left their bases in the north-east and violence initially fell, but revenge attacks quickly followed.
In June, Boko Haram carried out two attacks on schools in the region.
At least nine children were killed in a school on the outskirts of Maiduguri, while 13 students and teachers were killed in a school in Damaturu.
In July in the village of Mamudo in Yobe state, Islamist militants attacked a school's dormitories with guns and explosives, killing at least 42 people, mostly students.
Boko Haram regards schools as a symbol of Western culture. The group's name translates as "Western education is forbidden".
Boko Haram is led by Abubakar Shekau. The Nigerian military said in August that it might have killed him in a shoot-out.
However, a video released last week purportedly showed him alive.
Other previous reports of his death later proved to be unfounded.