Hello and welcome Anthony!!
I think you chose a great project to start with, even with the electronic fuel injection controls on this motor, it is as simple as it can get. I would say, start by reading the stickies and how-to write ups on this forum to learn what it takes to get involved with modifications and maintenance. Where replacement parts cannot be found, these writeups can guide you on how to install parts from other cars that were never meant to fit (say, a Miata radiator upgrade where more cooling performance is needed).
Being that you have an '84, the first thing that comes to mind is the timing chain. If your car has over 100k miles, and has not had the chain replaced at least once, it is definitely time to do it. Then you can do a dual-row timing chain mod, a kit you can get from LCE Performance or 22RE Performance, which makes the timing chain a lifetime piece. Both of those suppliers have good replacement and aftermarket upgrade parts for the engine. RockAuto can provide other stock parts. Techno Toy Tuning has suspension parts. ToyHeadAuto has a lot of upgrade parts as well, but for a premium. Buyee can guide you to Japanese auctions to import rarer parts that you'd want to get your hands on. Used parts, the Celica Facebook groups can lead you to folks with parts. I do hate to recommend the Facebook groups though, as it takes more focused discussion away from this forum.
I also have PDF's of the factory repair manuals and electrical diagrams for the car, please send me a PM with your email if you'd like me to send them to you!
Hope this helps, and let me know if you have any other questions!