There are only a few busses per day here so plan carefully. There is also a local train that goes to the nearby Nishi Yonezawa Station from which you could walk, but again, there are not many trains per day. A combination of the bus and train will likely serve you well to get there. You can also rent a bicycle for the day at Yonezawa Station and explore all the historical sites around Yonezawa too. I used the cycling option but took my own bicycle.
This is a fascinating castle to visit and it exemplifies some of the best in recent castle research. First, the trails are well maintained with signposts pointing out the different features of the castle. it is actually on private property, but the owner (whom I met) is very open to visitors and does a lot of work to keep up the site. The stonework around the masugata entrance was only discovered a few years ago and theories about the size and history of the castle are changing as people study the area more. As a result of these efforts and new discoveries, Tateyama Castle was named a National Historical Site in 2016.