Libosmscout in 2015

The end of the year (And the start of a new one, as I did not finish this article in time ;-)) is a good time to review the progress of libosmscout in the year 2015. As with likely every project at the end of the year one has the feeling, much more should have been done and there is even more left to be done in the next year – the more project time passes the more time is required to finish the project.

A retrospective is a good approach to correct this feeling and move things into a more objective perspective. After writing down all the things achieved in 2015 I’m of the opinion, that 2015 was a good year for libosmscout 🙂

What happened in 2015?

Bug fixes: As always 2015 brought a number of important bug fixes. Things didn’t work or broke – we fixed it.

Code cleanup and C++11 only: We did a huge number of code cleanups. Libosmscout is now C++11 only, requiring recent C++ compilers. On the other hand C++11 allowed us to (unconditionally) use things like unordered_map/set, lambdas, auto, range-based for loops and most important std::shared_ptr. Code got simpler and readable because of this. Using std:.shared_ptr allowed us to drop our own custom reference counting solution.

“Feature”-feature: Long planed and finally realized in 2015: The “Feature”-feature. Features allow the application developer to evaluate its own OSM attribute combinations, inject resulting data into the database and retrieve them back during rendering.
Libosmscout itself now uses features to map attribute information into the database. Obvious candidates for further custom Attribute are for example : ele, wikipedia links or similar.
Features also allow you to add custom labels and the rendering of multiple labels.

Disk size, performance, memory footprint: As every year 2015 also brought further optimizations of the on disk database format, resulting in less disk space been required and faster loading. We also did a number of performance optimizations, changing and improving the various indexes, data loading and rendering code. We also did improve the style sheet.
Compared to the performance at the beginning of the year we got measurably faster.
We also reduced the memory footprint by dropping some internal caches. Memory footprint however will increase again with more tile based caching and more caching in the renderer.

Automatic builds and more supported platforms: See the recent blog article regarding use of cloud based CI platforms. We now use Appveyor and Travis-CI to regularly builds libosmscout for various platforms.
The build in general was improved, too. Configure scripts have on one hand been improved and on the other hand been simplified. A global Makefile is available to build everything in one go.
2015 also brought Visual Studio 2015 support for the core libraries and tools and reintroduced MinGW/MSYS2 support.
Using above services allows use to better support these platforms, while spending less time for it. Thanks for the people at Appveyor and Travis-CI for their support for open source software! Note also the OpenHub web page for libosmscout.

CMake: We now have simple CMake support for all important sub projects. The support is not enough to actually build libosmscout but is a good start for anybody interested in further improving the CMake support.

Logging-API: We now have a simple logging API allowing us to redirect output in to to a file or some platform specific logging sink.

Better Location-API: Location API got some major bug fix resulting in better handling of regions. The API also got new methods for describing the current location.

Import of multiple data sets: The importer now allows import of multiple data files. This allows to enhance the database with additional information, one use case being addition of SRTM contours.

Conditionals in style sheet and “night”-mode: The style sheet was enhanced with new rendering options. Especially its is now possible to to use conditional statements. Conditionals can be influenced by the application code. One practical use was the additional of a (rudimentary) night mode variant for the default style sheet.

New tools: We added new tools (DumpOSS, performance tests) and improvements of existing debugging tools (DumpData).

More documentation: We added more documentation to the project. The installation documentation at the OSM wiki has been revised, new README files were added. Also inline source code documentation was improved.

OSMScout2: The OSMScout2 demo was enhanced, now looks nicer and better shows the libosmscout features.

Multi-threading and caching: The end of the year brought initial multi-threading support for database access. Most data files can now be accessed from multiple threads. A new cache based on tile cells was introduced, improving the database access time by better and faster caching. The cache also uses multiple threads to load and process data from different data files in parallel. 2016 will see more improvements in this area.

Finally I was able to join two hacker weekends organized and (payed) by the the German FOSSGIS e.V. Thanks for that. The participation allowed me to work an bigger features in one go, giving libosmscout a burst.

My thanks also to all the people that showed their interest in libosmscout by writing on the mailing list or connecting me privately. Some of them also offered patches and improvements. Please continue to do so.
I also encourage people interested in libosmscout or who are interested in general to work on new renderer implementations, routing or who would just like to try new things without starting from scratch, to contact the project. Libosmscout is not only a good choice for app development, but the amount of time already spent to define simple APIs and get things “fast” is also a good starting point for your experiments. Concentrate on your goals while using libosmscout. Libosmscout allows you to get to the point fast and concentrate on the things that are important for you. Please join, you are welcome!

So what is planed for 2016?

More optimizations: Of cause there are still ideas to make things faster. Libosmscout rendering is still a little bit too slow on older devices.

Multi-threading: 2015 will bring more support for multi-threading. More things will be don in parallel, more things will be callable from multiple threads.

More caching: The tiled data cache will get further improvements, allowing background loading of tile data. But I plan to work on caching of data in the renderer, too, likely improving rendering performance, too.

Improved data layout: Also on the list for 2016 are enhancements in the way the importer preprocesses data. Some planed changes will better allow using multiple databases for rendering and routing. Being able to embed node data into way and area data another point on the list. I also would like to experiment with enhance the data format to make use of complex ways consisting of multiple segments (with different attribute values), thus reducing the number of object to be rendered.

Support for Java using SWIG: I would also like to create an initial project setup for adding Java support via SWIG. If anybody has general SWIG project setup example code, please contact me.

More style options: After changing the renderer API to make direct use of tiling and adding support for caching within the renderer I would start to work on adding more features to the style sheet syntax and the renderer. Better label placement and rendering is high priority, but there other features like rendering labels on or beneath area borders, background images, stripes that are on the list.

Exception handling: I would like to switch to error handling via exceptions. The current solution to return bool results in ugly if-cascades and in general produces difficult to read code.

Borderlines: Better support for rendering of borderlines was also requested in 2015: IMHO the best approach is to generalize the approach implemented in the water index, allowing a more generic generation of indexes for similar cases.

Generation of mapping error reports: During processing a number of mapping errors are detected. This are complex things like resolving of relations or errors in the generation of the location index (most often spelling mistakes) but also simple errors occurring during parsing of individual attributes. Scanning these errors (and fixing them) can improve mapping and will (if feedback is given) also improve the quality of libosmscout.

Google Summer of Code 2016: Like nearly every other open source project libosmscout lacks people that actively participate on development. I’m in principle willing to support people that would like to work on libosmscout during the Google Summer of Code as mentor. Experience shows however that to make a good proposal and in result successfully get chosen, people should already have some experience with the project they would like to help out beforehand. IMHO in consequence now is the time to contact the project.


4 Responses to “Libosmscout in 2015”

  1. Der kleine Punk Says:

    wich repos is the newest github or sourceforge. I will build an Car App based on Raspberry and search lib to render the map

    • framstag Says:

      The official is the sourceforge repository. The github repository is sometimes more current, but is not guaranteed to be stable. Most of the time I use the github repository to trigger the central Travis and Appveyor builds to see, if my changes build.

      For questions please use the mailing list. For bug, issues and feature wishes you can use the mailing list or either the sourceforge or github bug tracker.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: