lostmapper.com

Getting Started with PostGIS in QGIS on macOS

Wed, 20 Dec 2023 00:00:00 GMT

What is PostGIS?

PostGIS is an extension for the popular relational database PostgreSQL. It adds support for storing, indexing and querying geographic data.

There are a number of reasons why you might want to use PostGIS but the primary advantage of using a database over files is the fact that multiple people can access, query and update information in a database at the same time.

Downloading and Installing Postgres.app

To get up and running fast we’re going to use an app called Postgres.app which includes PostGIS. You can download it from https://postgresapp.com

While this app provides a great way to start learning PostGIS, it’s likely not what you should use for an enterprise level application.

Head to https://postgresapp.com and download the latest release
Open the Disk Image and drag the Postgres application to your Applications folder
Start up the application from your Applications folder or Spotlight
Confirm that you want to open it
Click the Initialize button to create a PostgreSQL 16 instance
This will create a server with 3 databases:

your username
postgres
template1

Enabling the PostGIS Extension

Before we can do anything we first need to enable the PostGIS extension in our database.

Double-click the database named after your username and it will bring up a psql session in a terminal window. psql is a command-line tool for interacting with a Postgres database.

Enter the following commands to enable the PostGIS Extension on the database named with your username. In my case it’s named brian.

CREATE SCHEMA postgis;
CREATE EXTENSION postgis SCHEMA postgis;
CREATE EXTENSION postgis_raster SCHEMA postgis;
ALTER DATABASE brian SET search_path=public,postgis;
exit

Schemas are namespaces that help you organize the tables, views and other things you create in your database. We created a schema named postgis so that all the information associated with the PostGIS extensions won’t pollute or collide with other schemas we create. The extensions will still be available to all other schemas in the database.

Connecting to PostGIS from QGIS

Now that we have a database running with the PostGIS extension installed we can connect to it from QGIS:

Start up QGIS
In the Browser right-click on PostgreSQL and click New Connection
Enter the following:

Name: Postgres.app-username
Host: localhost
Port: 5432
Database: username

Under Authentication click Basic

User name: username

Click Test Connection
Click OK

Creating Schemas

As I mentioned previously, schemas are a convenient way to organize the tables, views and other database objects you create. Let’s create a dedicated schema called tutorial for the work we’ll be doing today:

In the Browser, under PostgreSQL, right-click on your new connection and choose New Schema…
Enter tutorial for the Name
Click OK

Creating Tables

Now that we have a dedicated schema, let’s create a table to hold some point data:

Right-click on the tutorial schema and choose New Table…
The Schema should already be set to tutorial
Change the table Name to points
Click Add Field
Change the field’s Name to name
Change the field’s Type to Text, limited variable length (varchar)
Change the field’s Length to 64
Set the Geometry Type to point
Click OK
Double-click the new points table in your tutorial schema to add it as a layer to your map

Adding Data to a Table

Once we have our points table as a layer in our project we can add data like we would with a GeoPackage or Shapefile:

Select the points layer
Click the Toggle Editing button to enable editing
Click theAdd Point Feature button to start adding points
Click on the map view to add a point
Leave the ID value as is — it’s automatically set to a function that grabs the next available ID for the table
Set a Name for the point
Click OK
Click the Save Layer Edits button
Right-click the points layer choose Open Attribute Table to see your points

Importing Layers into PostGIS via QGIS

Another thing we can do with PostGIS is import data from other formats, such as Shapefiles. Lets import that US States and Territories from weather.gov:

Download and unzip the US States Shapefile: https://www.weather.gov/source/gis/Shapefiles/County/s_08mr23.zip
In the Database menu choose DB Manager…
Under PostGIS choose the tutorial schema
Click Import Layer/File…
Click the … next to the Input field and navigate to where you unzipped the file s_08mr23.shp
Make sure Schema is set to tutorial
Change the Table name to us_states_and_territories
Check Primary Key and ensure it’s set to id
Check Geometry Column and ensure it’s set to geom
Check Convert field names to lowercase
Check Create spatial index
Click OK
Double-click the us_states_and_territories table under the tutorial schema in the Browser to add it as a layer to your map

Summary

You’ve now have a PostGIS database running on your machine and you know how to create your own schemas, create your own tables and import data using QGIS.

There are a couple of books you might want to check out — one I’ve been working through and the other I haven’t picked up yet:

If you’re interested in learning more please follow me on YouTube. If you have any questions or suggestions, feel free to reach out to me on Mastodon.

Setting Up Python 3 for GIS on macOS

Mon, 20 Nov 2023 00:00:00 GMT

From my explorations over the years it seems that Python is a particularly important language in the world of GIS. I thought it would be best to get comfortable with how to manage Python installations and the various external packages a project might use.

Install Python with asdf

I use several different programming languages in my day-to-day and after using various tools over the years have settled on using asdf to install and manage those languages.

asdf makes it easy to install multiple versions of a language and switch between them as your projects require. They provide extensive documentation on Installing asdf.

Once asdf is installed we can add its Python plugin, install the latest version of Python, and set that version as our global default:

$ asdf plugin add python
$ asdf install python latest
$ asdf global python latest
$ which python
/Users/brian/.asdf/shims/python
$ python --version
Python 3.11.5

Create a Project Directory

Now that Python is installed we can create a directory for our project:

$ mkdir lostmapper
$ cd lostmapper
$ asdf local python 3.11.5

Here we are making a directory for our project and switching into it. The last line creates a .tools-version file that lets asdf know what version of Python we want to use whenever we’re working in that directory.

Create and Activate a Virtual Environment

When you’re working many different programming projects they are likely to use both the same and different external packages. In some cases, they may even use the same package but a different version of that package. This can get confusing for both people and computers!

To avoid conflicts between the packages in any two projects it is best to give each project its own space to exist in. In Python, these are known as Virtual Environments and are managed using Python’s venv module.

To create and activate a virtual environment, use the following commands:

$ python -m venv .venv
$ source .venv/bin/activate

The first line creates a virtual environment in the subdirectory .venv. The second line activates that virtual environment. After that, any packages we add will only be installed within that virtual environment.

Don’t Check In Your Virtual Environment Directory

If you’re using a version control system like Git you’ll want to add the .venv directory to your project’s .gitignore file so that you’re not checking all those packages in.

Visualize Python Versions and Virtual Environments (Optional)

I use a tool called Starship that gives me a succinct and informational prompt in my terminal. It lets me know:

What directory I’m currently in
Which branch I’m on when using version control systems like Git
What languages and versions of languages are being used
What virtual environment I’m currently in, if any

For example, at this point, my prompt looks like this:

lostmapper.com on main via 🐍 v3.11.5 (.venv)

And yes, different languages have their own little emoji!

Add Packages Using pip

Now that we have a virtual environment set up we can start adding packages. To do that we’ll use pip — package installer for Python.

If we want to add the mkdocs package to our project, we would do the following:

$ python -m pip install mkdocs

If we look in .venv/lib/python3.11/site-packages we can see that mkdocs has been installed along with all the packages it depends on.

Freeze and Share Required Packages

Now that we’ve added some packages it’s a good idea to track what we’ve installed along with which specific version we’re using. This is helpful if we’re ever developing on another computer or collaborating with other people.

To generate a list of packages, run the following:

$ python -m pip freeze > requirements.txt

This creates a requirements.txt file in our project that we or others can use to install the same packages we’re using. The file could be named anything but requirements.txt seems to be the agreed-upon standard for Python projects.

Here’s what ours looks like now:

click==8.1.7
ghp-import==2.1.0
Jinja2==3.1.2
Markdown==3.4.4
MarkupSafe==2.1.3
mergedeep==1.3.4
mkdocs==1.5.2
packaging==23.1
pathspec==0.11.2
platformdirs==3.10.0
python-dateutil==2.8.2
PyYAML==6.0.1
pyyaml_env_tag==0.1
six==1.16.0
watchdog==3.0.0

Of course, depending on when you do this the version numbers may differ.

Installing Packages On Another Computer

To install all the same packages on another computer, run the following command after creating and activating your virtual environment:

$ python -m pip install -r requirements.txt

Summary

We’ve learned how to install Python using asdf, create virtual environments using venv, and add and track packages using pip. This should help us feel a little less lost when working on Python projects moving forward.

As one might guess, these are not the only tools available for managing Python versions and packages but for the most part, we’re leaning on what’s built in with Python — modules that should always be available once Python is installed.

If you’re interested in learning more please follow me on YouTube. If you have any questions or suggestions, feel free to reach out to me on Mastodon.

Getting Started with QGIS

Tue, 24 Oct 2023 00:00:00 GMT

What is QGIS?

QGIS is a free and open source Geographic Information System that runs on Windows, Mac and Linux. You can download it for free from https://qgis.org/

In this Getting Started guide I will show you:

How to create and save QGIS Projects
How to add a reference layer using OpenStreetMap
How to move and zoom around your map
How to create Spatial Bookmarks so you can quickly get to where you want to be
How to create a Print Layout of your map and export a PDF of it

For this article I will be using QGIS 3.32.3-Lima on macOS Sonoma.

Creating and Saving a QGIS Project

Most everything you do in QGIS is going to be done in the context of a Project, so it’s important that we get comfortable with creating and saving them.

Create a Directory for your Mapping Projects

I suggest keeping all your mapping projects together in a dedicated folder (or directory) on your machine. Depending on the size of your projects and data you may or may not want this folder to be in your iCloud Drive.

Determine where you want to store your projects
Create a folder named gisor mapping
Inside of that folder create a project folder — in this case we’ll call it interesting-places

Creating and Saving a Project

Now that we have a folder for our project, let’s create one in QGIS. There are a few ways we can do this:

Click the Project menu and select New
Click the New Project button in the Project Toolbar
Hold down Command and press N

Next we should give the project a formal title:

From the Project menu choose Properties…
Or press Shift Command P
Make sure the General tab is selected on the left
In the Project title text box enter Interesting Places
Click OK

We should now see that our QGIS window title has change from Untitled Project to Interesting Places.

Now that our project has a title we can save it:

Click on the Project menu and choose Save
Or press Command S
Navigate to the interesting-places folder you created above
Change the name in Save As if you’d like
Click Save

Adding an OpenStreetMap Layer

You may have noticed that the main “document” area of QGIS is currently blank and gives us no idea where we are mapping. Fortunately we can use OpenStreetMap to provide a nice base map to quickly visualize things:

In the Browser panel on the left open up XYZ Tiles
Right-click on the OpenStreetMap entry and choose Add Layer to Project
You can also just double-click the OpenStreetMap entry to add it to your project

Panning and Zooming around a Map

When you start QGIS the default tool that’s enabled is the Pan Map tool — the hand icon in the Map Navigation Toolbar. You can click and drag to pan around the map but it’s not too helpful when we’re this zoomed out.

Let’s get a closer look at things by zooming in:

Switch to the Zoom In tool (the magnifying glass with the plus sign inside it)
Click and drag a rectangle around the area that you are interested in

Using the Locator Bar

Another way we can quickly find what we are looking for is by using the Locator Bar in the lower left of the QGIS window:

For example, if we wanted to find Easter Island we could use the Nominatim Geocoder option to find it:

Click on the Locator Bar (or press Command K)
Type the great than symbol > followed by a space and the words easter island
Use the arrow keys to navigate the entries
Press Return to select the highlighted entry
You can also just double-click on the entry you are interested in

You can then use the Zoom In tool to further refine the area you are looking for:

Creating Spatial Bookmarks

As we work maps we will likely be panning and zooming in and out. It would be convenient to be able to return to this initial view of the island after we’ve completed some detail work.

To do that we can create what’s known as a Spatial Bookmark. These bookmarks can be stored at a Project level or a User level. User level Spatial Bookmarks are good for locations you might need to refer to across several projects. In this case we’ll create a Project level Spatial Bookmark. Once again, there are a few ways to accomplish this:

In the Browser panel open up Spatial Bookmarks, right-click on Project Bookmarks and choose New Spatial Bookmark…
Or, on the View menu select new Spatial Bookmark…
Or, press Command B

Whichever method you choose you will be greeted with a **Bookmark Editor **dialog. For this bookmark we’ll set the following:

Name: Easter Island
**Saved in: **Project Bookmarks

Leave everything else as is and click Save. You should now see your new bookmark show up in the Browser under Project Bookmarks.

If we pan and zoom around the map and ever want to get back to this initial view, all we need to do is double-click the Easter Island bookmark.

This bookmark will also come in handy when creating a Print Layout, which is what we’ll do next.

Creating a Print Layout

Up until now we’ve been working in what’s called the Map View. This view is a bit of a mixture between managing geographic data and styling geographic data.

When we want to create a printable view of our map we need to create a Print Layout. A Print Layout will allow us to add things like titles, legends, north arrows, attributions, area maps and things of that nature. It will also let us save the resulting layout as a PDF and other image formats.

Adding a Print Layout

To create a new print layout:

Click on the Project menu and choose New Print Layout… (or press Command P)
In the Create Print Layout dialog provide a title for your layout. We’ll call ours Landspace

Setting the Page Size

Before we add any elements to our print layout we need to set the Page Size. This is easiest to do when there’s nothing else on the map yet:

Right-click on the blank layout and choose Page Properties…
On the right side in the Item Properties tab we can set the Size and Orientation

For my layout I’ll be setting the Size to Letter and the Orientation to Landscape

Adding a Map Title

Let’s add a title to our map so folks know what we’re showing:

In the Toolbox on the left click Add Label
Click and drag a rectangle out at the top of the page
In the Item Properties on the right to set the Text, Font, Horizontal Alignment and whatever other settings pique your interest

Adding the Map

Of course our map isn’t much use with the actual location on it, so let’s add it:

Click on the Add Map tool in the Toolbox
Click and drag a large rectangle out for the map
If the correct area is not showing you can use the Bookmarks button in the Item Properties panel to select the Spatial Bookmark we made previously — another place it comes in handy!

It might seem silly to have to add the actual map to the layout but there are cases where you might want to have multiple maps in a layout, such as having a detailed view with an inset overview of the overall location.

Saving a PDF

The last step is to export our map as a PDF. Would you be surprised to learn there’s a couple of ways to do this?

From the Layout menu choose Export as PDF… or from the Layout Toolbar choose Export as PDF…
Select the location and name you want to save the PDF as
Click Save
In the PDF Export Options you can leave everything as is and click Save

The only thing left to do is open up your PDF and check it out.

Summary

This is only scratching the surface of what is possible with QGIS but what you’ve learned today will be the basis of many mapping projects in QGIS.

If you’re interested in learning more please follow me on YouTube. If you have any questions or suggestions, feel free to reach out to me on Mastodon.