Blog

Viewing posts by Tim Sutton

Does hosting your own server still make sense in 2020?

Posted by: Tim Sutton | 1 bulan, 2 minggu ago | Comments
Client: We need a server for this project and we want to host it in house
Me: Why?
The procurement of a server / multiple servers represents a large capital outlay. If you amortize that outlay over a 5 to 10-year lifespan, in my opinion, it is usually a poor investment. With cloud hosting, your equipment will be hosted in a facility with proper cooling, dust management, and power management - all critical for the long reliable operation of a server. Unless your project partners have facilities like this in place already, you can expect higher than average hardware failure rates and server downtime. In addition, a cloud provider has strong physical security, removing the possibility that equipment can be stolen or reappropriated.
If there is a failure on a cloud-hosted machine, they will have spare supplies on hand, or they can swap out the entire server in a short space of time, getting you back into production ASAP.
Additionally, if you invest capital in a server, the capacity of your storage and computing power is constant throughout the project. If you opt for cloud servers, you can scale up and down your infrastructure based on demand and usage patterns, saving financial and environmental costs.
Many cloud providers are now also running their data centers on green electricity (for example our preferred provider https://hetzner.com).
Using a cloud server approach allows you to deploy your software using a ‘one server, one task’ approach where you deploy many small servers, each optimized for the workload needed. This removes the single point of failure outcome of deploying a single monolithic server and also prevents one wayward job on a server crippling all the other jobs on the server.
Cloud service providers also provide great failure support for things like nightly and ad-hoc server snapshots which image the whole server. The server can then be restored from that snapshot as a running instance. You can also create off-server and cross data-center backups.
For traffic management and network load cloud providers offer the ability to deploy into different regional data centers so that the traffic can be spread around the internet. Combined with using a content distribution network (CDN) like Cloud Flare, they can reduce the effects of latency introduced by far-away and heavily used sites.
The need for specialized staff to maintain servers is radically reduced when the server is cloud-hosted (in which case only a sysadmin or developer role is needed) versus on-site which will often also introduce the need for hardware technicians skills.
Lastly, I will mention that for most applications, the most common of which is a general-purpose web site, the argument that a server needs to be close-by for good performance is largely irrelevant since browsing web pages is generally not latency critical and a few milliseconds difference in response times will be unnoticed by users. In our company, almost all the servers we have deployed for African projects are located in Germany and the clients would be hard-pressed to notice the different response times wise.
There are a few critical things to consider when cloud hosting which may be used as arguments why cloud hosting is not suitable:
  • Some governments (e.g. Canada, Indonesia, others) have legislated that government datasets must be hosted on servers within the physical borders of the country. My response to this is usually to seek our local cloud providers. For example, Amazon has data centers in Canada that may be used for the Canadian govt. services.
  • The security requirements of the organization mandate that data may not leave the confines of the organization. In this case, the procurement of physical servers and hosting on-site may be a prerequisite. My comments about cooling, power supply, dust management, and physical security should be borne in mind then.
  • Cloud server costs can get away from you if not carefully managed. Some cloud vendors are, in my opinion, better suited for their ability to scale and bring online/offline computing resources than they are for hosting long-running servers. Amazon was an example of this last time I looked into it. Hetzner gives pretty good value, even for long-running servers.
  • Cloud servers may experience contention issues. In a project we have with a US-based client we were finding database performance very slow. It turned out that a lot of other servers using the same shared physical storage were adversely affecting disk performance.

Creating a live, topic specific mirror of OpenStreetMap in PostGIS

Posted by: Tim Sutton | in Docker, PostGIS, QGIS | 1 tahun, 3 bulan ago | Comments

Introduction

In this article, we will do a walk-through of creating a live mirror of OSM for a specific country or region and for a specific set of OSM features. For this example, we will fetch all building data for Angola into a PostGIS database, and update that database with new features as they arrive in OSM. Providing an OSM mirror in this way is a powerful tool to pair the power of OSM with the power of QGIS and PostGIS. Now you will be able to do offline analysis of the data in OSM. How does this all work? Here is a little diagram that illustrates the underlying architecture (click for a larger version):

FBIS 2 Launch Workshop

Posted by: Tim Sutton | 1 tahun, 3 bulan ago | Comments

How to make beautiful lollipop call out labels in QGIS

Posted by: Tim Sutton | in QGIS | 1 tahun, 3 bulan ago | Comments

Call out labels are a handy cartographic instrument for attaching labels to features on the map where you want the label to be offset from the feature being labelled. It allows you to prevent the map becoming overcrowded. I call the variant I describe here 'lollipop' labels because the 'callout line' is rendered with a decorative ball at the end.

Using NOTIFY to automatically refresh layers in QGIS

Posted by: Tim Sutton | in PostGIS, QGIS | 1 tahun, 4 bulan ago | Comments

One of the most brilliant but little-known features of QGIS is the ability to trigger layer refreshes and events in response to notifications from PostgreSQL. This was developed by the wizards from Oslandia and is easily added to any existing table in your PostgreSQL database - including PostGIS tables. This feature was added in version 3.0 (see https://qgis.org/en/site/forusers/visualchangelog30/#feature-trigger-layer-refresh-or-layer-actions-from-postgresql-notify-signal). Take for example this simple table:

How to easily add South African and Namibian toposheets as XYZ tiles to QGIS

Posted by: Tim Sutton | in QGIS | 1 tahun, 6 bulan ago | Comments

Thanks to the great work of Grant Slater and the OpenStreetmap team, there are freely available XYZ tilesets for South Africa and Namibia 1:50 000 series toposheets which can easily be added to QGIS. Here is the general procedure to add a layer:

Using a SAMBA/CIFS mount as a docker volume

Posted by: Tim Sutton | in Docker | 2 tahun, 4 bulan ago | Comments

I preface this article by saying that what I am showing here is probably not best practice and you should test to see if it works reliably for you before taking this route. I should also mention that this is a hacky approach because it breaks our ‘never log on to a server’ rule, so it is not a long-term solution - it is a short-term hack until Hetzner adds a cloud storage offering to their cloud platform.

How to resolve issues with QGIS 3 Processing Dialog Layouts on MacOS

Posted by: Tim Sutton | in QGIS | 2 tahun, 10 bulan ago | Comments

If you are using QGIS 3 master builds on MacOS and encounter issues with the display of processing dialog layouts like this:

Kartoza at GeoNode Summit 2016

Posted by: Tim Sutton | in GeoNode | 3 tahun, 8 bulan ago | Comments

Four of the Kartoza team (Tim, Etienne, Ismail and Rizky) attended the GeoNode Summit in Rome held in November 2016. The World Food Programme provided the venue for the event and our attendance was funded by the WorldBank/GFDRR. The event was a really great opportunity to interact with the GeoNode community. We used the opportunity to learn about the upcoming plans for GeoNode and discuss some of our own plans. 

Have any questions? Contact us.
We would love to hear from you!
Subscribe to our Mailing list (spam-free, industry-relevant occasional emails)

Subscribe to our mailing list

* indicates required