Now that I've unlocked everything in Sayonara, and I've seen a substantial amount of the other two games I feel I can weigh in on this one.
Sayonara is my favourite by far. Its physics feel the most polished to me. I like the practice first / survival later mindset. Because I could explore branches as I liked, I always had new things to try. I didn't have to repeat early levels a thousand times to get another crack at the harder ones. It's also the least random; every level seems fair to me. Getting some optimal things might rely on random/chaotic things, but there's always a safe and predictable way through, I think. I like that it has extra characters (featuring alternate soundtracks from previous games), but especially checkpoints help with training. I don't think these games have great bosses, but Sayonara's are the most fun. The crab boss concept in particular I think is marvellous; I love the idea of dropping persimmons on it.
Shun is my least favourite for being on the opposite side of the same reasons. Not being able to expand/explore the map outside of survival mode makes trying new fields a tedious task. It's made worse because it's got a kind of arcade/sadistic streak. There's a lot of randomness, aside from random spawns, things like the flying squid bosses can be extremely unpredictable (I still don't have a good coping strategy for them). Some tasks, while not random, are inherently chaotic, so they're just as good as random. For example: traversing a conveyor ceiling against its flow, and then having a pinpoint accuracy task at the end of that traversal- there's just no way to consistently swing to the same spot. Add time limits to this, and also that the hard levels are buried long into the run, and it's just not very fun trying to unlock new fields that way. I'm still enjoying it, but I feel that it's working against me in this way.
Umihara Kawase is somewhere in the middle. The hard levels aren't as sadistic as Shun's, and it seems less chaotic in general. It has the best music of the series, and I'm not sure if I like its art or Sayonara's best (kind of a tie). The bosses are tedious, but I'd gladly take them over Shun's shorter but randomly behaving bosses. I really miss a dedicated button for diagonal throws, too, like the other two games have. It also needs an in-game map, but it's easy enough to print one out.
I think the bottom line is that the most fun I have is getting to new stages, and only Sayonara really supports that goal. It's the best one to learn on. Shun has the most challenge. Even though I like Sayonara the best, I'd still rather move on and play new fields in the other games than replay "better" fields in Sayonara.

Fairness/non-chaotic: Sayonara, Umihara, Shun
Bosses: Sayonara, Umihara/Shun (tie: boring vs chaotic)
Challenge: Shun, Sayonara, Umihara
Music: Umihara, Sayonara*, Shun (* Sayonara has Umihara and Shun's music in it too.)
Art: Umihara/Ayonara, Shun