Month: August 2020

This work by Collabora Productivity was possible thanks to AMD.

LibreOffice 7.0, just released, includes a new drawing backend based on the Skia library, which allows LibreOffice to use the modern Vulkan API to graphics operations. This Visual Class Library (VCL) backend is the default on the Windows platform, superseding the OpenGL-based backend.

Working on the future of graphics for office productivity

Having multiple VCL backends has its benefit to integrate with different operating systems, but each backend performing its own rendering implementation is far from optimal, since we cannot add new rendering functionality and assume that it will work cross-platform. Many backends across different platforms and toolkits miss optimized paths for various rendering tasks. This adds complexity and makes it hard to ensure that rendering objects happens in an accelerated way everywhere.

Another problem is that multiple backends regularly perform the same type of mapping from VCL’s APIs to what a modern toolkit provides these days. This duplication means not only maintenance cost, but also can lead to having to fix the same bug at multiple places.

Moving away from GDI and OpenGL

The VCL library is responsible for widgets (buttons, controls, etc.) and basic rendering. It does not implement the drawing directly, but it provides an internal API, which is implemented by various backends that implement the actual graphics operations. These backends usually adapt LibreOffice to each platform , for example the ‘win’ backend is used on Windows, the ‘kf5’ and ‘gtk3’ backends are for Unix-like platforms using the KDE Frameworks and the Gtk3 graphics toolkit respectively and there is a ‘headless’ backend used by tests that does not render to the screen.

Each VCL backend uses an underlying graphics API available on the platform to perform the graphics operations. The Cairo library is used by some Unix-like and ‘headless’ backends, the ‘osx’ backend uses the macOS Quartz. On Windows, the ‘win’ backend has several plugable drawing implementations:

• GDI drawing. This code is relatively old and has several limitations, such as not being double-buffered.
• OpenGL drawing. This code provides GPU-accelerated drawing. OpenGL is used directly, so all relevant code including OpenGL shaders needed to be implemented in LibreOffice. The OpenGL API is also slowly being phased out in the industry.
• Skia drawing. The Skia library hides implementation details, and provides several rendering methods, including Vulkan API.

Integrating Skia and Vulkan

The Skia library is not shipped in a binary form, not even by 3rd-party providers such as Linux distributions. It is provided only as source and the usual way to use it in a project is to ship it with that project. LibreOffice includes Skia as a 3rd-party library and fortunately building it using the LibreOffice gbuild system is reasonably simple. An additional complication is that Skia provides a new release roughly every 6 weeks, and generally only the latest release receives any fixes, requiring repeated updates. Since Skia is continually evolving, each update may also require adjustments (although so far it seems they are generally small).

Collabora’s developers taking the first hurdles

The Skia API is generally well documented, but it appears to be mostly aimed at developers already working on a project using Skia, such as Chrome. Important classes have their API well described, but it can be difficult to find tutorials for them and some classes are harder to understand at the beginning (for example, SkPixmap, SkBitmap, SkImage and SkSurface are all classes representing a drawable, but it was unclear at the beginning what the suitable use cases would be for each for them).

Similarly, it is not obvious how to use Skia in a new project. There does not seem to be any actual developer introduction to Skia in the documentation, nor does thereseem to be any documentation on how to setup a new or standalone project using Skia. There is a Hello-World example in the examples/ directory that is based on platform integration code that is not part of the Skia library itself but is inside a sk_app/ directory in the tools/ directory. That code was usable for LibreOffice, but required patching to cover LibreOffice needs (for example, the code created a new graphics context for each toplevel window, and Skia requires proper graphic context to be used in drawing operations, but LibreOffice code sometimes does not know which toplevel window will use the result of a drawing).

LibreOffice is a fairly old codebase, and still uses relatively old concepts such as paletted bitmaps, low-resolution bitmaps (such as 4bpp) or drawing that uses the XOR operation on pixels. Skia, being a relatively new library, has features that assume modern concepts are used (for example, RGB bitmaps are required to be 32bpp with unused alpha channel, 24bpp RGB bitmaps are not supported).

Once the initial learning period is over, Skia is consistent in its API and reasonable flexible to use allowing progress to be relatively fast. Code using Skia is very readable, and using Skia makes future maintainance tasks easier to perform. Even some old concepts, such as LibreOffice using a separate bitmap for alpha channel, sometimes interpreted as 8bpp and sometimes as alpha, later blended with data bitmap to get the actual result, could be mostly implemented using Skia API, making them GPU-accelerated.

Nice performance and even room for improving

While there were some concerns about how performance would be affected by moving to Skia compared to OpenGL driver and and hardware implementations that have been heavily optimized over the past several decades, it turns out that performance within LibreOffice is at least equivalent to the OpenGL version and synthetic benchmarks show that there is room for improvement.

Benefits for LibreOffice technology

While somewhat complicated at the beginning, using Skia in LibreOffice has been in general a rather pleasant experience. In the future LibreOffice’s use of Skia could be extended to other platforms, reducing the number of platform rendering APIs used, eliminating duplicated code, reducing bug count and generally improving quality.

For comparison, LibreOffice OpenGL drawing code is roughly 12k lines of code, while Skia drawing code is only 4k.

Apart from the immediate benefits, moving to Skia and Vulkan on Windows paves the way for a single, powerful, hardware-accelerated rendering API cross-platform.

Moving to Skia on Windows required about 7 person-months ol, which lets us use Vulkan acceleration without large implementation costs. It will be interesting to see how much time is saved in the next few years from the reduced maintenance cost. The resulting work is mature enough that there is no real negative change in performance, and we have not started heavily optimizing yet.