As you are already aware, there will be a big national holiday week from the end of April to 6th May in Japan, which is called "Golden week".
During the Golden week, it is expected that tickets for bullet trains will be in high demands. Most of bullet trains have non-reserved carriages too, however, you may have to stand on the train. We would therefore, suggest you to reserve bullet train tickets in advance to avoid heavy congestion. A bullet train ticket is available up to 1 month prior.
If you will use a JAPAN RAIL PASS（JR Pass）, however, online reservation is only available with JR East:
It means that if you are a JAPAN RAIL PASS holder, you cannot make online reservation for a bullet train between Tokyo and Kyoto/Osaka, for example. In that case, you can only book a bullet train seat once you get to Japan. For the detailed booking process with a JR Pass, please visit the official website below: