"Cascade Point" by Timothy Zahn (1983, won the 1984 Hugo award) contains fictionalized mathematical analysis of higher-order dimensions of space/time.

The novel concerns future space travel whereby spacecraft enter a higher dimension using a field generator, then rotate the spacecraft within the field which results in translational displacement of the craft (i.e. it travels forward in space). The spacecraft then stops rotating, turns off the field generator, and the craft is then back in normal space, several light years away from where it started the maneuver.

The mathematical analysis comes in to play when they passengers and crew of one starship find that they have inadvertantly entered one of the branching dimensions and must find their way back to their original dimension. There are numerous references to mathematical models of "cascade theory", Riemann surfaces, curve fitting of statistical and empirical data, geometric intuition, and the many problem inherent in trying to visualize or develop visual representations of mulitple dimensions.

This is a very fun read, in the tradition of techie space opera. There's a strong female character (how often does that happen?) and a few tidbits (pages 14 and 49) of complex analysis on Riemann surfaces/branch cuts. The main SF thing here is alternate universes, and the branch cuts for the aforementioned Riemann surfaces correspond to the boundaries between these universes.

Please note that Timothy Zahn's "Cascade Point" has been re-anthologized in his "Pawn's Gambit and other Stratagems" 2016.