There are broadly two approaches to handling rich text in Headless Wagtail:
Rendering on the backend
Wagtail stores rich text internally in a HTML-like format with some custom elements to support internal page links and image embeds. On a traditional Wagtail site, those custom tags are converted into standard HTML, but this doesn't happen in the built-in API. The API returns unprocessed rich text content, which means that users need to either parse the HTML on the frontend and convert the custom tags or they need implement a custom serializer to render the RichText fields on the backend. Currently, wagtail-grapple pre-renders the HTML. So one solution could be to update the built-in API to also pre-render the HTML (see for example PR #8695).
Rendering on the backend is currently the easier approach to user for managing rich text. Note that using this approach requires page URLs to follow Wagtail's conventions, so custom routing isn't possible without some complex configuration.
Rendering on the frontend
Pre-rendering the HTML on the backend may be more convenient if you're happy with the way Wagtail renders it, but it's still difficult to customise the rendering on the frontend. Other headless CMSs provide Rich Text as a sequence of blocks in JSON format. This approach makes it easier to customise the rendering of the blocks without having to find a way to parse the HTML fragment.
This approach is currently the harder approach for managing rich text. A long-term solution for this may involve creating a JS library or plugin.