Adding to pressure to the outlook is China's virus outbreak, which has threatened exports and factory output and has already hit tourism in Japan.
Gross domestic product (GDP) is expected to have contracted an annualised 3.7% in the October-December quarter, the poll found, having grown 1.8% in the third quarter.
It would be the first contraction in the five quarters and the biggest fall since a 7.4% decline in April-June 2014, which was the last time Japan raised its sales tax.
The expected annualised contraction would translate to a 0.9% quarter-on-quarter decline after the economy grew 0.4% in the third quarter, the poll showed.
"Manufacturers' production and earnings were weak due to falls in exports, and employment and wages recovery slowed. We see the trend of consumer spending was weak," said Kentaro Arita, senior economist at Mizuho Research Institute.
"Exports and capital spending are expected to stagnate and consumer spending will stay weak. We expect economic growth in the first half of 2020 will be limited considering the coronavirus impacts."
Private consumption, which accounts for over 50% of GDP, likely dropped 2.0% for the quarter, the first fall in five quarters and the fastest decline since it fell 4.8% in April-June 2014.
Capital spending was seen down 1.6% in the fourth quarter, the first fall in three quarters and the biggest since a 3.4% drop in July-September 2018.
External demand — or exports minus imports — likely contributed 0.3 percentage point to GDP growth in the final quarter of 2019, the poll showed, although that positive contribution is mostly due to weakening imports rather than export strength.
It subtracted 0.2 percentage point off GDP in the third quarter last year.
The Bank of Japan's corporate goods price index (CGPI), which measures the prices companies charge eachother for goods and services, likely rose 1.5% in January from a year earlier, led by price gains in oil related products, the poll found.
Japanese Prime Minister Shinzo Abe has ordered his government to take "all necessary steps" to mitigate the impact of the virus outbreak on the economy, including tapping state budget reserves.