It is the latest act of provocation by the communist state since it fired missiles into the Sea of Japan three days ago, following comments from President Donald Trump that he would meet any threats from North Korea with "fire and fury".
If initial reports prove correct and a long-range missile has been flown over the Japanese mainland, this will be regarded as a far more provocative act than flying shorter-range weapons into the sea.
The Japanese government's J-Alert warning system advised people in the area to take precautions, but public broadcaster NHK said there was no sign of damage.
Japan's military did not attempt to shoot down the missile, which passed over the country's territory at around 6:06 am. local time (21:06 GMT), reportedly flying 2,700 km (1678 miles).
The country's state media reports the missile broke into three pieces off the coast of Hokkaido.
Japan's Prime Minister Shinzo Abe said he would everything in his power to protect the public.
"We will make utmost efforts to firmly protect the lives of the people," Mr Abe told reporters as he entered his office for emergency meetings on the missile firing.
South Korea also said the North had fired an unidentified projectile early on Tuesday from a region near its capital, Pyongyang, eastwards towards the sea.
Seoul's Joint Chiefs of Staff said the South Korean and US militaries were analysing the launch and did not immediately confirm how far the projectile travelled and where it landed.
Japan's Chief Cabinet Secretary Yoshihide Suga said North Korea's latest missile launch was a threat that Tokyo would respond to decisively.
"This ballistic missile launch appeared to fly over our territory. It is an unprecedented, serious and grave threat to our nation," Mr Suga told reporters.
He added that Japan will work closely with the US, South Korea and other concerned nations to take a timely and appropriate response.
"It could endanger peace and security in the Asia-Pacific region. It is also very dangerous and problematic in terms of the traffic safety of planes and ships. The launch is an obvious violation of UN resolutions. We cannot tolerate these repeated provocations by the North. We condemn this in the strongest possible way."
The launch comes days after the North fired what was assessed as three short-range ballistic missiles into the sea and a month after its second flight test of an intercontinental ballistic missile, which analysts say could reach deep into the US mainland when perfected.
North Korean leader Kim Jong Un threatened to fire missiles into the sea near the US Pacific territory of Guam and expressed anger over the ongoing annual war games between the United States and South Korea. A missile would need 3,500 km of range (2,175 miles) to hit Guam.
South Korean and US forces are in the middle of their annual joint exercises, with Pyongyang regularly complaining that these are an invasion rehearsal.
US and Japanese servicemen have just concluded joint exercises in Hokkaido, Japan's northernmost major island.
The Pentagon confirmed that the missile test was carried out over the Tohuku region in the north.
"The missile launched by North Korea flew over Japan," Pentagon spokesman US Army Col. Rob Manning said. "We are still in the process of assessing this launch."
A senior US intelligence official told NBC this would be the first missile test to pass over Japan on a high altitude trajectory. In 1998, North Korea fired a missile through Japanese airspace.
The UK Foreign Secretary Boris Johnson said on Twitter he is "outraged'' by the "reckless provocation'' from the latest missile launch.
The Japanese government is expected to be prepared to ask the UN for assistance in dealing with the provocation from North Korea.
Update your news preferences and get the latest news delivered to your inbox.