It is always an exciting time to see freshly minted ARM based silicon arriving in the form of Apple’s massive shift to the ARM based M1. This of course means work for Collabora’s LibreOffice team too. The code needs to be prepared for M1, step by step. Here we update you on the status of the work, and what needs to be done.
With the launch of the new Apple devices nearing, it is important that suitable software arrives around the same time as new hardware. Apple ensures this by a translation layer, so that software for Intel Macs can be used, using Rosetta translation.
Nevertheless, given the code size of LibreOffice, for the best performance it makes most sense to have a pre-optimized native binary. As such Collabora joined the Universal App Quickstart Programme back in July and has been doing work on enabling LibreOffice for M1 since then.
This effort is made possible by the kind support of those who buy LibreOffice Vanilla in the Mac app store. Thank you! And thanks too to Tor Lillqvist for his patience and hard work here.
The status of the work
All of these changes are in master, or in the gerrit queue getting past our CI automation:
Configuration changes (mostly there). It should now possible to configure and build a native LibreOffice on a Apple Silicon, as well as cross-compiling to x86_64.
Patching and fixing of lots of bundled libraries to make them build cleanly.
Then there is a first attempt at a new C++/UNO ABI bridge – we need to match Apple’s ABI by tweaking Linux’s ARM64 support to match. This allows UNO scripting to work (in theory).
We still have some failing unit tests, that need investigation, as well as some other bits, described below.
All of this means that LibreOffice should start and work on M1! So far it has had only very basic Writer & Calc testing. The more exciting, complex features are not yet tested.
What’s next .. want to get involved?
Post-launch, if you can get an M1 Mac, then help is always most welcome! We have several missing pieces that will require further work, with some unusual low-level bits.
The new C++/UNO ABI bridge requires more testing, to ensure the UNO scripting support works smoothly.
Enabling bits we didn’t compile in yet: Firebird, Java (when there is a JDK).
Scripts to combine builds for arm64 and x86_64 into one universal app (i.e. one where binaries are “fat,” consisting of separate parts for each architecture)
Adaptation to whatever new checks are added for universal apps in the App Store. This is an ongoing unpredictable part of our work: adapting 8 million lines of code to the latest updated rules, keeping our builds compiling and signing with the latest tool chains.
Of course we’ll continue to work to bring the best LibreOffice possible to Apple Silicon as time permits, and we are confident that even if we don’t make it in the next month or two, Rosetta 2 will fill the gap using dynamic instruction set translation. And when all is done, our open source desktop productivity tools will benefit the full power of the new Macs!
Autumn is just around the corner. For many participants in the GSoC 2020, a busy and instructive summer full of hacking on open source projects came to an end a few weeks ago. Commits have been contributed and final reports have been written. This year experienced Collabora Productivity developers were again mentors for various projects of the Google Summer of Code for the LibreOffice project. Here are some examples of projects our team helped to succeed!
Analysing Writer documents with the “Style Inspector”
The “Style Inspector” is a great new tool. You can access it through a new icon (an eye combined with a pencil) in the Sidebar (also via “Sidebar Settings”). The Style Inspector displays in full detail (and hierarchical) all styles and also direct formatting applied to a cursor position in a Writer document. So you can analyse, identify problems and clean them up. Sometimes formatting in documents is messy and people mix styles with direct formatting. The Style Inspector allows you to see that.
The feature is available for testing in pre-released development versions of LibreOffice. Shivan Kumar Singh picked up the proposal from the LibreOffice Design Team. He was mentored by Collaborans TomažVajngerl and MikeKaganski with Heiko Tietze from the LibreOffice Design Team. Take a look into Singh’s final report! It is an inspiring a guide on how to approach a big project like LibreOffice.
Improving the way to find and add extensions
There are many useful extensions to LibreOffice and users should be able to find them easily! Like in app stores like Gnome Software or the Play Store. That is basic idea behind “Tight Integrations” and the proposal from the LibreOffice design team. Yusuf Keten made this his GSOC project a success and added the possibility to search and sort the through extensions without having to leave LibreOffice. You start this new way to search with a clear yellow star with a download arrow, that is in the templates dialog, at the icons in the view options, or from the galleries pane in the side bar.
If you are curios about this handy extensions feature you can already test and find it in the latest LibreOffice pre-releases. Yusuf was mentored ba Collabora’s Muhammet Kara and Heiko Tietze from the LibreOffice design team. Find all the details of Yusuf’s work in his final report.
“I learned a lot of things during the GSoC. Although GSoC is finished, I will continue to contribute to LibreOffice. I am very happy to be part of the LibreOffice community.” (Yusuf Keten)
Changing the contour – shadows are becoming blurry
Did you know, that in LibreOffice the shadows are just a copy of the object? There are already a lot of settings to change their appearance, like its colour, its angle, the transparency and distance behind the object. Mentored by Collaboran’s TomažVajngerl and Miklos Vajna, Ahmad Ganzouri added another option. The “Blurry Shadows” make use of the already implemented BitmapFilterStackBlur and make the shape of the shadow look very realistic. Find the details around the development in Ahmad’s final report. We have seen the Blurry Shadow option in the master branch and expect it to be available to all users in Version 7.1 of LibreOffice. The option can be easily accessed via the “Area” dialog in “Objects & Shapes” or directly via the corresponding Sidebar module.
Searching for a mentor? Join us GitHub!
Google Summers of Code are an excellent opportunity to learn working in many open source projects. But where to find mentors during the rest of the year? We recently moved the code of Collabora Online to GitHub. You will find a growing community there, with easy hacks to get started. Community Mentor Muhammet Kara and the rest of our team of open source developers are there and willing to share their vast experience.
This work by Collabora Productivity was possible thanks to AMD.
LibreOffice 7.0, just released, includes a new drawing backend based on the Skia library, which allows LibreOffice to use the modern Vulkan API to graphics operations. This Visual Class Library (VCL) backend is the default on the Windows platform, superseding the OpenGL-based backend.
Working on the future of graphics for office productivity
Having multiple VCL backends has its benefit to integrate with different operating systems, but each backend performing its own rendering implementation is far from optimal, since we cannot add new rendering functionality and assume that it will work cross-platform. Many backends across different platforms and toolkits miss optimized paths for various rendering tasks. This adds complexity and makes it hard to ensure that rendering objects happens in an accelerated way everywhere.
Another problem is that multiple backends regularly perform the same type of mapping from VCL’s APIs to what a modern toolkit provides these days. This duplication means not only maintenance cost, but also can lead to having to fix the same bug at multiple places.
Moving away from GDI and OpenGL
The VCL library is responsible for widgets (buttons, controls, etc.) and basic rendering. It does not implement the drawing directly, but it provides an internal API, which is implemented by various backends that implement the actual graphics operations. These backends usually adapt LibreOffice to each platform , for example the ‘win’ backend is used on Windows, the ‘kf5’ and ‘gtk3’ backends are for Unix-like platforms using the KDE Frameworks and the Gtk3 graphics toolkit respectively and there is a ‘headless’ backend used by tests that does not render to the screen.
Each VCL backend uses an underlying graphics API available on the platform to perform the graphics operations. The Cairo library is used by some Unix-like and ‘headless’ backends, the ‘osx’ backend uses the macOS Quartz. On Windows, the ‘win’ backend has several plugable drawing implementations:
GDI drawing. This code is relatively old and has several limitations, such as not being double-buffered.
OpenGL drawing. This code provides GPU-accelerated drawing. OpenGL is used directly, so all relevant code including OpenGL shaders needed to be implemented in LibreOffice. The OpenGL API is also slowly being phased out in the industry.
Skia drawing. The Skia library hides implementation details, and provides several rendering methods, including Vulkan API.
Integrating Skia and Vulkan
The Skia library is not shipped in a binary form, not even by 3rd-party providers such as Linux distributions. It is provided only as source and the usual way to use it in a project is to ship it with that project. LibreOffice includes Skia as a 3rd-party library and fortunately building it using the LibreOffice gbuild system is reasonably simple. An additional complication is that Skia provides a new release roughly every 6 weeks, and generally only the latest release receives any fixes, requiring repeated updates. Since Skia is continually evolving, each update may also require adjustments (although so far it seems they are generally small).
Collabora’s developers taking the first hurdles
The Skia API is generally well documented, but it appears to be mostly aimed at developers already working on a project using Skia, such as Chrome. Important classes have their API well described, but it can be difficult to find tutorials for them and some classes are harder to understand at the beginning (for example, SkPixmap, SkBitmap, SkImage and SkSurface are all classes representing a drawable, but it was unclear at the beginning what the suitable use cases would be for each for them).
Similarly, it is not obvious how to use Skia in a new project. There does not seem to be any actual developer introduction to Skia in the documentation, nor does thereseem to be any documentation on how to setup a new or standalone project using Skia. There is a Hello-World example in the examples/ directory that is based on platform integration code that is not part of the Skia library itself but is inside a sk_app/ directory in the tools/ directory. That code was usable for LibreOffice, but required patching to cover LibreOffice needs (for example, the code created a new graphics context for each toplevel window, and Skia requires proper graphic context to be used in drawing operations, but LibreOffice code sometimes does not know which toplevel window will use the result of a drawing).
LibreOffice is a fairly old codebase, and still uses relatively old concepts such as paletted bitmaps, low-resolution bitmaps (such as 4bpp) or drawing that uses the XOR operation on pixels. Skia, being a relatively new library, has features that assume modern concepts are used (for example, RGB bitmaps are required to be 32bpp with unused alpha channel, 24bpp RGB bitmaps are not supported).
Once the initial learning period is over, Skia is consistent in its API and reasonable flexible to use allowing progress to be relatively fast. Code using Skia is very readable, and using Skia makes future maintainance tasks easier to perform. Even some old concepts, such as LibreOffice using a separate bitmap for alpha channel, sometimes interpreted as 8bpp and sometimes as alpha, later blended with data bitmap to get the actual result, could be mostly implemented using Skia API, making them GPU-accelerated.
Nice performance and even room for improving
While there were some concerns about how performance would be affected by moving to Skia compared to OpenGL driver and and hardware implementations that have been heavily optimized over the past several decades, it turns out that performance within LibreOffice is at least equivalent to the OpenGL version and synthetic benchmarks show that there is room for improvement.
Benefits for LibreOffice-technology
While somewhat complicated at the beginning, using Skia in LibreOffice has been in general a rather pleasant experience. In the future LibreOffice’s use of Skia could be extended to other platforms, reducing the number of platform rendering APIs used, eliminating duplicated code, reducing bug count and generally improving quality.
For comparison, LibreOffice OpenGL drawing code is roughly 12k lines of code, while Skia drawing code is only 4k.
Apart from the immediate benefits, moving to Skia and Vulkan on Windows paves the way for a single, powerful, hardware-accelerated rendering API cross-platform.
Moving to Skia on Windows required about 7 person-months ol, which lets us use Vulkan acceleration without large implementation costs. It will be interesting to see how much time is saved in the next few years from the reduced maintenance cost. The resulting work is mature enough that there is no real negative change in performance, and we have not started heavily optimizing yet.
Collabora developers are mentoring students during this year’s Google Summer of Code
Google Summer of Code is an amazing, international program encouraging the participation of university students in open source software development. It enables students to independently gain practical experience on concrete projects and matches them with experienced developers. This year a record number of almost 1200 students is taking part in the sponsored projects. Collabora is always delighted to support GSoC projects: some of our experienced developers are mentoring the students: guiding them in the code, encouraging them, and supporting the growth of their skills.
Collabora developers at GSoC 2020
Also for this year’s GSoC several students are being mentored by Collabora Productivity team members. Tomaž Vajngerl, Muhammet Kara, Miklos Vajna, Mike Kaganski, and Michael Meeks are assisting the work on several projects around LibreOffice. And our companies involvement is not just limited to that. Andrej Shadura, a developer from our parent company Collabora, is in charge of various projects around the Debian operating system. Collabora’s support of these Google Summer of Code reflects our commitment to open source software and is also a matter of personal attachment. Several of today’s Collaborans like Elie Tournier and Rohan Garg have successfully participated in the GSoC program in the past. So have the current mentors Muhammet Kara, Miklos Vajna, and Pranam Lashkari from Collabora Productivity team.
A success story – development camps in Turkey
The successful application of no less than six LibreOffice office projects for this year’s Summer of Code would not be fully told without mentioning some previous happenings in winter. Collabora Productivity had sponsored the LibreOffice Developer Bootcamp in Ankara and the LibreOffice Development Workshop at the Anadolu University in Eskişehir (Turkey) during the Free Software Wintercamp 2020. These ambitious development courses by Muhammet Kara spawned no less than three of the six successfully submitted LibreOffice projects of GSoC 2020. Find out more about the great outcome of these development camps within our community news blog post.
LibreOffice projects mentored by Collabora Productivity
The student Ahmad Ganzouri has successfully submitted two projects. Supported by the mentors Tomaž Vajngerl, Miklos Vajna, and Michael Meeks, he will work on a project adding blurry shadows to objects in LibreOffice and on a second one that aims to move the current gallery data in favour to easier to handle ZIP files. Yusuf Keten, another participant of the LibreOffice Developers Bootcamp in Ankara, will be working on additions to better integrate extensions. Keten’s project is being mentored by Collabora’s Muhammet Kara.
Also Collabora’s parent company developers are mentoring
Another interesting GSoC project is bringing a CiviCRM integration to LibreOffice and Collabora Office. “Styles Inspector“, also mentored by Tomaž Vajngerl, is another project aimed at LibreOffice. We all look forward to the results of the students work. And for sure we will try to cover some of that here. Another wonderful story is that of Collabora Productivity developer Pranam Lashkari. He was a GSoC student during the last two years while graduating and this year he is already a mentor for two projects within the Boost C++ libraries. Collabora developer Andrej Shadura is mentoring no less than three projects focusing on Android SDK Tools for the Debian operating system. Find out more about these and other noteworthy projects in the Collabora news blog.
About Google Summer of Code
Google Summer of Code is a program awarding a three months stipend to students who successfully complete a free and open-source software coding project during the summer. The annual program started in 2005. This year, 199 organizations and 1199 student projects are taking part. Within the last 15 years more than 15.000 students from over 100 countries had been accepted. Developers from free and open source projects serve as mentors. Between 80 – 90% of the accepted projects get completed successfully.
It’s now easier to check the status of your /libreoffice/online patches
Recently the CI (continuous integration) infrastructure for Online has been made accessible via the internet. Now developers from outside Collabora can directly check the status of their patches and builds.
Already for a while there was a Jenkins instance that checks patches on gerrit for the online-project (and our other projects), before they are submitted to cgit.freedesktop.org/libreoffice/online/
This service is now running on dedicated hardware, what allows us to make it accessible directly from the internet. Links pointing to this server in TDF gerrit will work from now on.
How to participate in the project
This growing access to our development makes it easier to join the project. Which we of course encourage 🙂 Therefore… read more about participating in developing LibreOffice online here!
If you have any questions, please leave a message here.