"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Group the data by class\n",
"grouped = data.groupby('CLASS')\n",
"\n",
"# Let's see what we have\n",
"grouped"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can see, `groupby` -function gives us an object called `DataFrameGroupBy` which is similar to list of keys and values (in a dictionary) that we can iterate over.\n",
"\n",
"Check group keys:"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"dict_keys([32111, 32112, 32200, 32421, 32500, 32611, 32612, 32800, 32900, 33000, 33100, 34100, 34300, 34700, 35300, 35411, 35412, 35421, 36200, 36313])"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"grouped.groups.keys()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The group keys are unique values from the column by which we grouped the dataframe.\n",
"\n",
"Check how many rows of data each group has:"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Terrain class: 32111\n",
"Number of rows: 1 \n",
"\n",
"Terrain class: 32112\n",
"Number of rows: 1 \n",
"\n",
"Terrain class: 32200\n",
"Number of rows: 2 \n",
"\n",
"Terrain class: 32421\n",
"Number of rows: 110 \n",
"\n",
"Terrain class: 32500\n",
"Number of rows: 2 \n",
"\n",
"Terrain class: 32611\n",
"Number of rows: 257 \n",
"\n",
"Terrain class: 32612\n",
"Number of rows: 11 \n",
"\n",
"Terrain class: 32800\n",
"Number of rows: 80 \n",
"\n",
"Terrain class: 32900\n",
"Number of rows: 28 \n",
"\n",
"Terrain class: 33000\n",
"Number of rows: 5 \n",
"\n",
"Terrain class: 33100\n",
"Number of rows: 118 \n",
"\n",
"Terrain class: 34100\n",
"Number of rows: 3005 \n",
"\n",
"Terrain class: 34300\n",
"Number of rows: 1 \n",
"\n",
"Terrain class: 34700\n",
"Number of rows: 3 \n",
"\n",
"Terrain class: 35300\n",
"Number of rows: 134 \n",
"\n",
"Terrain class: 35411\n",
"Number of rows: 35 \n",
"\n",
"Terrain class: 35412\n",
"Number of rows: 449 \n",
"\n",
"Terrain class: 35421\n",
"Number of rows: 5 \n",
"\n",
"Terrain class: 36200\n",
"Number of rows: 56 \n",
"\n",
"Terrain class: 36313\n",
"Number of rows: 8 \n",
"\n"
]
}
],
"source": [
"# Iterate over the grouped object\n",
"for key, group in grouped:\n",
"\n",
" # Let's check how many rows each group has:\n",
" print('Terrain class:', key)\n",
" print('Number of rows:', len(group), \"\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are, for example, 56 lake polygons in the input data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also check how the _last_ group looks like (we have the variables in memory from the last iteration of the for-loop):"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" GROUP | \n",
" CLASS | \n",
" geometry | \n",
" area | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4303 | \n",
" 64 | \n",
" 36313 | \n",
" POLYGON ((377127.305 6688073.257, 377116.045 6... | \n",
" 9619.307973 | \n",
"
\n",
" \n",
" | 4304 | \n",
" 64 | \n",
" 36313 | \n",
" POLYGON ((371141.897 6677999.999, 371139.757 6... | \n",
" 25266.167705 | \n",
"
\n",
" \n",
" | 4305 | \n",
" 64 | \n",
" 36313 | \n",
" POLYGON ((371498.720 6680399.799, 371497.585 6... | \n",
" 364.087680 | \n",
"
\n",
" \n",
" | 4306 | \n",
" 64 | \n",
" 36313 | \n",
" POLYGON ((375668.607 6682942.062, 375671.489 6... | \n",
" 2651.800270 | \n",
"
\n",
" \n",
" | 4307 | \n",
" 64 | \n",
" 36313 | \n",
" POLYGON ((368411.063 6679328.990, 368411.424 6... | \n",
" 376.503380 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"**String formatting**\n",
" \n",
"There are different approaches for formatting strings in Python. Here are a couple of different ways for putting together file-path names using two variables:\n",
"\n",
"```\n",
"basename = \"terrain\"\n",
"key = 36200\n",
"\n",
"# OPTION 1. Concatenating using the `+` operator:\n",
"out_fp = basename + \"_\" + str(key) + \".shp\"\n",
"\n",
"# OPTION 2. Positional formatting using `%` operator\n",
"out_fp = \"%s_%s.shp\" %(basename, key)\n",
" \n",
"# OPTION 3. Positional formatting using `.format()`\n",
"out_fp = \"{}_{}.shp\".format(basename, key)\n",
"```\n",
" \n",
"Read more from here: https://pyformat.info/\n",
"
\n",
"\n",
"\n",
"Let's now export terrain classes into separate Shapefiles.\n",
"\n",
"- First, create a new folder for the outputs:"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Results folder exists already.\n"
]
}
],
"source": [
"# Determine output directory\n",
"output_folder = r\"L2_data/\"\n",
"\n",
"# Create a new folder called 'Results' \n",
"result_folder = os.path.join(output_folder, 'Results')\n",
"\n",
"# Check if the folder exists already\n",
"if not os.path.exists(result_folder):\n",
" \n",
" print(\"Creating a folder for the results..\")\n",
" # If it does not exist, create one\n",
" os.makedirs(result_folder)\n",
" \n",
"else:\n",
" print(\"Results folder exists already.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"At this point, you can go to the file browser and check that the new folder was created successfully.\n",
"\n",
"- Iterate over groups, create a file name, and save group to file:"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Saving file terrain_32111.shp\n",
"Saving file terrain_32112.shp\n",
"Saving file terrain_32200.shp\n",
"Saving file terrain_32421.shp\n",
"Saving file terrain_32500.shp\n",
"Saving file terrain_32611.shp\n",
"Saving file terrain_32612.shp\n",
"Saving file terrain_32800.shp\n",
"Saving file terrain_32900.shp\n",
"Saving file terrain_33000.shp\n",
"Saving file terrain_33100.shp\n",
"Saving file terrain_34100.shp\n",
"Saving file terrain_34300.shp\n",
"Saving file terrain_34700.shp\n",
"Saving file terrain_35300.shp\n",
"Saving file terrain_35411.shp\n",
"Saving file terrain_35412.shp\n",
"Saving file terrain_35421.shp\n",
"Saving file terrain_36200.shp\n",
"Saving file terrain_36313.shp\n"
]
}
],
"source": [
"# Iterate over the groups\n",
"for key, group in grouped:\n",
" # Format the filename \n",
" output_name = \"terrain_{}.shp\".format(key)\n",
"\n",
" # Print information about the process\n",
" print(\"Saving file\", os.path.basename(output_name))\n",
"\n",
" # Create an output path\n",
" outpath = os.path.join(result_folder, output_name)\n",
"\n",
" # Export the data\n",
" group.to_file(outpath)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Excellent! Now we have saved those individual classes into separate Shapefiles and named the file according to the class name. These kind of grouping operations can be really handy when dealing with layers of spatial data. Doing similar process manually would be really laborious and error-prone."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Extra: save data to csv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also extract basic statistics from our geodataframe, and save this information as a text file. \n",
"\n",
"Let's summarize the total area of each group:"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
"area_info = grouped.area.sum().round()"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"CLASS\n",
"32111 1834.0\n",
"32112 2148.0\n",
"32200 105737.0\n",
"32421 702073.0\n",
"32500 109747.0\n",
"32611 13135597.0\n",
"32612 107343.0\n",
"32800 1465278.0\n",
"32900 617209.0\n",
"33000 659465.0\n",
"33100 3777595.0\n",
"34100 12381611.0\n",
"34300 1627.0\n",
"34700 2786.0\n",
"35300 1382940.0\n",
"35411 411198.0\n",
"35412 4710133.0\n",
"35421 67864.0\n",
"36200 9986966.0\n",
"36313 43459.0\n",
"Name: area, dtype: float64"
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"area_info"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- save area info to csv using pandas:"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [],
"source": [
"# Create an output path\n",
"area_info.to_csv(os.path.join(result_folder, \"terrain_class_areas.csv\"), header=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"In this tutorial we introduced the first steps of using geopandas. More specifically you should know how to:\n",
"\n",
"1. Read data from Shapefile using geopandas\n",
"\n",
"2. Access geometry information in a geodataframe\n",
"\n",
"4. Write GeoDataFrame data from Shapefile using geopandas\n",
"\n",
"5. Automate a task to save specific rows from data into Shapefile based on specific key using `groupby()` -function\n",
"\n",
"6. Extra: saving attribute information to a csv file.\n",
"\n",
"\n",
" "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}