China wants the yuan to become an alternative global reserve currency to the US dollar.
Last year, Beijing and Tokyo agreed to promote the direct exchange of their currencies to cut costs for businesses and boost bilateral trade.
China and Japan have the world's largest foreign exchange reserves.
"For China, the move is linked to its efforts to internationalise the yuan," said Zhang Yongjun of the China Centre for International Economic Exchanges.
"Allowing foreign investments in its debt market will make the yuan more accepted internationally." China does not allow foreign investors to freely purchase government bonds. Beijing has also maintained a tight control over foreign investments in yuan-denominated assets.
In an indication that it was willing to loosen its grip, last month Beijing allowed E Fund Management, one of China' s largest asset managers, to offer its Hong Kong customers the option of investing in the yuan directly by buying stocks and bonds on the mainland.
However, the scheme has an investment quota of 20bn yuan ($3.2bn; £2bn), which many observers say limits its reach.
While the move to allow Japan to purchase its bonds is another major step, analysts said that Beijing needs to loosen its controls further if it wants the yuan to be recognised as a reserve currency.
"It will only truly be an international currency if foreign central banks can freely buy and sell it according to their needs," Michael Pettis of Peking University told the BBC.
"So far what we have are inter-government agreements which are just cosmetic."
However, some analysts said that a global role for the yuan could only be achieved if the Chinese currency became fully convertible.
They said such a move would require China to open up its financial and capital markets, something that Beijing has been reluctant to do.
"It is difficult to have an international acceptance for your currency if you cannot put it in and take it out of the country with relative ease," said Patrick Chovanec of Tsinghua University.
Mr Chovanec explained that if China opened up its capital markets, it may make it difficult for Beijing to simultaneously control exchange and interest rates as it does now.
"This means it makes it harder to use the yuan as a policy tool," he added
China's currency policy has been a contentious issue. China's major trading partners have accused Beijing of keeping the value of the yuan artificially low in a bid to boost its exports.
While China has allowed the yuan to appreciate against the US dollar in recent months, it has warned that a sudden rise will not only hurt its export sector but also have a negative impact on its overall economy.