Thanks you so much, fellas. It's really awesome to get all the help and input from you all.

So I had a look in the fuse box under the hood. This is what I found when I removed the 15A horn/hazard fuse:

And my following reaction:

I replaced the fuse and plugged everything in. And voila! You'll also notice that the radio bezel doesn't really fit around the faceplate... yeah.

The horn and hazards worked again, and the radio powered on! I plugged my phone in via the aux jack and played music. It sounded... interesting, given how tiny the speakers are (what, 3 1/2"?). I played around with the fader/balance, and found that the front left speaker would play audio even if I biased the audio only at the rear right speaker. So something's a little wonky there but at least everything functioned! Just for shiggles, I plugged in the factory radio...

And it all worked!

Granted, I was missing a light on the right-hand-most equalizer knob, the left/backward tune button and [1] memo button didn't work, and there was some scratching in the audio from using the fader/balance knobs. But it worked!!! The tape player worked a little better, it didn't eat the tapes and actually played audio, but the audio cut out and came back periodically.
It's boring to look it up in a book, but that's what I do. Fuse chart and tables for a 1984 Celica scanned into pics, going back to the topic of your head unit.
The horn/hazard circuit just keeps the head unit's memory alive for radio station presets and the like. I'd think operating power should come from the switched 12v. Does your antenna actually work, wires okay and all? I was surprised at how little (as in zero) radio reception I got with the power antenna down vs extended. Antennas, who knew?
Thanks for the diagrams! My antenna mast is actually bent up and I don't know if the motor actually works or not.. I got decent radio signal even with it in its partially extended state.
In my experience, most car radios won't run without both the constant 12V and switched 12V power.
The hazard/horn fuse also has something to do with the headlight retract relays, so replacing it should also help with that.
Well, that's what ended up happening!
---
I haven't looked at the thermostat yet, that will be next on the list. I'll remove it and try the pot-of-hot-water test to see if it actually opens at the specified temperature. I'm
pretty sure my radiator has something to do with it, the fins inside are all brown. Brown brown brown brown brown...
More to come!