Archive for category: Development_en

You are here: / Development_en
ChristianK

Major changes to the mapping process

, ,

The entire production process for OAM maps has become somewhat outdated, so a number of upgrades to the development and production environments have become necessary. The Linux environment within Windows (WSL2) has been migrated from Debian 11 to Ubuntu 24.04 LTS, which has far-reaching consequences.

1.) Python 2 is dead

  • This means that all Python scripts must be converted to Python 3; this affects:
  • The software that generates the contour lines had to be replaced = these may differ very slightly from the previous ones in terms of curve rounding.
  • The script that transfers the Mapsforge POI files to Locus, using the pyspatialite library among other things, had to be rewritten to use mod_spatialite.
    Explanation: SQLite3-Spatialite is used for fast proximity searches for POIs in the apps.
  • 90% of the Python OSM tools now need to be moved into an isolated Python environment (venv) = everything needs to be set up from scratch, including the GDAL bindings.
  • The script that calculates the sea/land areas from Jochen Topf’s SHAPE files has been replaced.

2.)
The alignment and modification of the original OSM data via OSMOSIS TagTransform now takes 1–2 hours (!!) for average-sized maps alone, and significantly longer for the large Germany maps or the Alps. Steffen, a C++ programmer who has often helped me out of a tight spot, has now made available a new C++-based TagTransform that is significantly faster – but may also contain some initial bugs….

3.)
The OSM PLACE=* relations, which I had largely eliminated after converting them to NODES, I was forced to reactivate in some cases, which has led to massive side effects in terms of rendering times – these have now multiplied for some maps – on average, a complete map set now takes a good five days longer to complete, which is rather annoying.
Here I have to painstakingly debug again to find out which giant relations are causing this (99% of which are completely unnecessary).

 

If there are any issues with the next updates (including the one coming in April), please let us know in the forum.

Wishing you all a lovely summer,
the OpenAndroMaps team

/5 Comments/by
ChristianK

September update released

, ,

In short:

 

The resolving of the SuperRoutes for the hiking trails is finished – similar to the cycle superroutes, which takes us into the waters of the route rules of Lonvia/Waymarkedtrails sind, as far as possible.

The resolution of the super routes has been rewritten to the best of my knowledge and belief – but I have to say that not every creativity in the OSM data mischmasch can be taken into account.

 

An interesting development is the new class of long-distance/rapid cycle routes
The internal discussion with all background information (mixed German/English) can be found here.

 

Also the render themes by Tobias = Elevate/Elements (Forum)
and those from
JohnPercy = VoluntaryUK (Forum) and Website

have been further improved – e.g. the contour lines are now rendered as curves.

 

In addition, it should be said that very extensive changes in the map structure were necessary and that it is possible that small errors have crept in. Please do not hesitate to report such errors in the forum.

 

We wish you beautiful tours in autumn
the team of OpenAndroMaps

/by
ChristianK

Outlook for the September update

, ,

What’s coming up at OAM:

The resolving of the SuperRoutes for the hiking trails is finished – similar to the cycle superroutes, which takes us into the waters of the route rules of Lonvia/Waymarkedtrails sind, as far as possible.

 

An interesting development is the new class of long-distance/rapid cycle routes that are being built in NL/DE/BE/USA and also mapped in OpenStreetMap as cycle_highway=yes. These expressways are less for leisure than for commuting/commuting/shopping.
Therefore, in future we will display the two types of cycle paths (leisure versa commuting=rapid cycle routes) independently from each other and, if applicable, on a separate layer, depending on the render theme.

 

The preparations for this from our side are already in the late beta stage and we will, if everything runs well, put a new map set with the corresponding render themes online at the beginning or mid of September.

The internal discussion with all background information (mixed German/English) can be found here.

 

Have a nice summer
The Team of OpenAndroMaps

/by
ChristianK

New Maps / New Features

, , ,

And there we go,

 

As all those who have ever met me probably guessed, my sense of order is stronger than my frustration and I have written a script for the resolution of the screwed-up super routes and integrated it into the process.

 

And because I enjoyed that typing so much, I changed the generation of the REFs (= short names of the routes).
Until now, the REF – if there were capital letters in the NAME and no REF-Value – was formed from these.
So e.g. “West Coast Trail” = WCT
Unfortunately, there are also relations which, especially in Scandinavia, had the path name_+vegur +vegen etc. in the NAME. This ended up in a REF that only consisted of the first capital letter.

From now on, the capital letters only become effective when at least TWO of them are recognised.
If there is only one capital letter, the first 5 letters of the NAME are used to form the REF.
So “Fjärilsleden” was REF=”F” until now, from now on REF=”Fjäri” – looks much, much better.

 

I did allow myself a new POI, namely the “E” charging stations relevant for EBikes without the obligation to consume (Pub,Restaurant…).
These can be found in the POIs under hike/bike, I think this is also of interest to one or two of you.
Here is the exact implementation: https://www.openandromaps.org/oam-forums/topic/ebike_charging_public-2

 

Perspective: In the course of the discussion about the ICN/IWN + super routes, it came up again that the “big” routes cover the small, local cycling/walking routes too often – we are currently discussing possibilities to switch off the international routes, but this is an essential intervention in the structure of the maps and must be considered = this will take a little time.

 

So, I wish you a pleasant Summer

Best regards,
Christian

/8 Comments/by
ChristianK

USA+CDN now with 1″ Contourlines

,

Viewfinderpanoramas.org has released high resolution 1″ elevation data for North America.
This means that for the USA and Canada a new OpenAndroMaps map set with more accurate contour lines is now available.

The file sizes have of course increased but the added value of information for hikers and cyclists outweighs this by far.
Especially ditches and ridges can be estimated much better with the 30m grid of the 1″ data than with the previous 90m grid of the 3″ elevation data.

The following maps were split to keep file sizes in reasonable limits:

BritishColumbia complete > BC/North + BC/South
UtahColorado > Utah + Colorado
CarolinaGeorgiaFlorida > CarolinaGeorgia + Florida

 

Best regards, Christian

/1 Comment/by
ChristianK

Locus: too much Elevate Themes in Themeswitcher

,

Hi, we recently changed the scipted delivery of the Elevate/Elements Theme while installing the maps from this website.

Up to now the Elevate Theme was downloaded, unzipped and installed to [internalMemory]/Locus/mapsVector/_themes/elevate/
From now on the Theme is downloaded and the zip file copied to [internalMemory]/Locus/mapsVector/_themes/ AND it is not unzipped couse Locus can handle the Theme as zip much faster than a bunch of scattered grapfics.

Reason: Android11 is dead slow while reading lots of graphics but can handle them fine if inside a zip-file.
Noone knows why they have changed this, but – yeah….

So: if you already have the Elevate Theme as unzipped version in [internalMemory]/Locus/mapsVector/_themes/elevate/
AND
Install the new, faster, zipped Android 11 Elevate Theme you have:

TWO Elevate and TWO Elements Themes in your Locus Themeswitcher.
Nice, indeed – and I cant solve this by scripting cause I can’t delete files on your divices.

However: Its easy to solve this problem:
Install a FileManager like the Totalcommander https://play.google.com/store/search?q=totalcommander&c=apps , go to [internalMemory]/Locus/mapsVector/_themes/ and completely delete the “Elevate” folder.

Edit: With new Android Versions it’s possible that the theme is in the PrivateAppFolder, or wherever ……

 

Have a nice summer
Christian

/0 Comments/by
ChristianK

New Render-PC in operation, new V5-Beta-Maps

,

Many thanks to all the donors who
made the new equipment possible

The new rendermachine is running since this morning and is currently being fed with the latest Planet-File. The aim is to put new maps under the Christmas tree for our community. At the same time, these maps are being completely renovated.

Test maps and a beta version of the new Elevate redertheme are available here:

https://www.openandromaps.org/oam-forums/topic/elevate-updates-and-test-versions-news/page/5/#post-45699

Whats new, in a nutshell:

  • The MTB routes are now rendered independently and by category (like the cycle routes) – this means that supra-regional MTB routes such as in the UK or the Granittrail in the Waldviertel are highlighted prominently.
  • Overlaps between bike and MTB routes are now neatly resolved
  • Short names of MTB/bike/hike routes are now separated, this is only now possible with the fast rendermachine and Mapsforge V5
  • The connections between route nodes (NL) are now rendered later at higher zoom levels for better overview
  • Dozens of OSMC hiking symbols are added
  • etc..
/0 Comments/by
ChristianK

Excel becomes LibreOffice-Calc at OAM

, ,

Many people might ask themselves: “Why Excel for an OSM map project?
Well, with OAM there is a stock of around 270 maps to manage with the most diverse settings.

For example, map symbols for petrol stations and drinking water sources in urban areas appear only at much higher zoom levels than in Scandinavia – for this, there is also the ELEMENTS theme by Tobias, which takes this into account. This means that different tag mappings have to be created and assigned to individual maps.
But also such trivial things as map centres, languages, fineness of elevation data, allocation to render queues (load balancing) have to be managed. Corresponding start-up programmes for each individual production step such as conversion of file formats, creation of boundary polys, elevation data, sea areas, inheritance of bike/MTB/hike routes etc,etc. – belong to !automated! production.

Last but not least, the web pages must be regenerated after each update.

This is impossible to do manually, so far Excel-VBA programmes in combination with PERL scripts have done most of the hard work.

 

For about 10 days now, I have been sitting in front of the computer every free minute and porting VBA to Starbasic. Porting means cleaning up 9 years of organically grown programmes and simply rewriting them, leaving out my beloved PERL for pure administration jobs.
In addition, a flexible folder structure was implemented to allow porting to other machines and effective load balancing/caching.

 

With the switch to a free office package and the cleaning up of the programmes, it is now much easier for my co-authors to familiarise themselves with map production in OpenAndroMaps. 

 

The first maps rolled out today from the LibreCalc generated start scripts, the automated creation of the download pages will be addressed next week.

 

New rendering machine: Thank you very much for your donations – most of the parts for the PC have arrived and are awaiting assembly, which will be started after the next map update if there are no unexpected problems with the software port.
For me this is a great reassurance as it means that the test computer will go to @Michael and thus 10 years of accumulated knowledge about vector maps will be preserved.

 

There will be more detailed information about the new render machine.
Here in a nutshell: AMD Ryzen 5600G, 128GB RAM, ASUS MB, 2TB 970evo Plus as standard drive, 1TB Toshiba SSD for the planet extracts and height data, NOCTUA fans, new semi-passive power supply, as well as various used parts from the spare and new fans for the old case.
Should my time run out, this computer will also go to @Michael.

best wishes and stay healthy
Christian

 

/1 Comment/by
ChristianK

New System für Test and Development

,

Dear OAM community,

Summer is slowly giving way to a colorful autumn and I have a new test system on my desk.
The old “system”, an ancient Lenovo T410 from the stone age has really become too small, even for testing purposes, and often goes into thermal emergency shutdown.
So I bought a used ASrock Deskmini which was used as a game server.
With a super-efficient Intel i3-8100T and 64gb of (new) RAM as well as some older spare parts plus a few new parts, a quite performant system was created, which is also quite environmentally friendly and quiet.
The 500€ that this system cost (260€ only the RAM) came mostly from reserves of your donations.

What is currently happening is the conversion of all scripts to Windows10 and here to a large extent to WSL, the integrated Linux environment of WIN10 which, believe it or not, is actually quite mature. Now finally the, often buggy and specially adapted, Windows versions of Phyghtmap, osmconvert, osmfilter and especially everything based on GDAL/Python – can run native and always UpToDate in the Linux shell of Win10.
The effort for the conversion is unfortunately considerable, Python2 scripts e.g. do not run any more and must be rewritten, Windows has also naturally again some new bugs.

A word to the users in the forum: I have made myself a bit rare in the forum, unfortunately my resources are limited due to my health. To say it clearly: My arteries are unfortunately quite dense.

Therefore, as soon as the conversion to Win10/WSL is completed, the production system will be updated to a new motherboard + an AMD RYZEN + 128gb RAM and the development system will be transferred to the production machine.
Unfortunately we still need about 1200€ – so if you have some money left over from your christmas bonus it would help us a lot 😉 .

The development computer will be handed over to @Michael who has been developing the OAM world maps for years and as an old IT-professional has the best prerequisites.
Michael will familiarize himself with the (by no means trivial) creation of the OAM maps and, in the long run, will also take over this task.

Health and nice autumn tours
Christian and the small team of OpenAndroMaps

/1 Comment/by
ChristianK

More maps with 10m Contourlines

, ,

The elevation data for Europe are now very accurate in many areas thanks to flight laser scanning.
Wherever these data are freely available, @Sonny collects them, checks them and interweaves them with Jonathan de Ferranti‘s Viewfinder panoramic elevation data in the areas not yet scanned.

The result is a constantly growing set of elevation data, the best that could be achieved without paying licence fees at the moment.
The accuracy is good enough to make sense for contour lines at 10 meters vertical distance. (which was definitely not the case so far).

To make a long story short:
The following maps are now available with 10m contour lines (more will follow):

Alps, Alps_East, Alps_West, Pyrenees, Austria, CanaryIslands, Corse, Czech_Republic, Great_Britain, UK_Scotland, Ireland, Madeira, Mallorca_Ibiza, Malta, Sardegna, Sicilia, Slovakia, Slovenia, Switzerland, Luxembourg, UK_Wales, UK_LakeDistrict, UK_IsleOfMan, UK_Shetlands

Baden-Wuerttemberg, Bavaria

 

 

New and updated Maps:

The Alps-West now also includes the neighbouring Rhone Valley.
New map: BalticStates (Estonia, Latvi, Lithuania as complete map)

/1 Comment/by
© Copyright - openandromaps 2012-2022