Get the "Applied Data Science Edge"!

The ViralML School

Fundamental Market Analysis with Python - Find Your Own Answers On What Is Going on in the Financial Markets

Web Work

Python Web Work - Prototyping Guide for Maker

Use HTML5 Templates, Serve Dynamic Content, Build Machine Learning Web Apps, Grow Audiences & Conquer the World!

Hot off the Press!

The Little Book of Fundamental Market Indicators

My New Book: "The Little Book of Fundamental Analysis: Hands-On Market Analysis with Python" is Out!

Who Has Power and Who Doesn't? Mapping The World Electrical Power Plants with Python and Basemap

Introduction

Let's use two great, frequently curated data sets to map the power plants from around the world by both country and population and identify who still doesn't have access to this essential and life-altering source of energy.



If you liked it, please share it:

Code

Global Energy - Who Has IT?
In [124]:
from IPython.display import Image
Image(filename='../energy for all.png', width='60%')
Out[124]:

Who Does and Doesn't Have Access to Electricity Around the World?

Hello Freinds,

Let's talk about power! More precisely, electrical power! I'm going to walk you through two great, frequently curated data sets and we'll create a series of maps to see who has and who doesn't have access to electrical energy.

In [146]:
Image(filename='../world-capacity.png')
Out[146]:

Tell me this, what would go through you head if you just hauled you luggage to your hotel room and couldn't get connected to the interent using the supplied address and password, or worse, the place doesn't offer wifi??? You'd wonder what antiquated century you just walked into... in my family, it would be an instant mutiny, they'd probalby have me walk to the plank if i couldn't find a close by internet cafe or even another hotel...

But did you konw on this planet, 1 in 10 of our bothers and sisters, right now, don't have access to electricity??? It is mind boggling - wifi, for those unfortnuate folks is the least of their worries, they're like stuck on a never ending camping trip, a very rough and unsafe one...

Less that 1 Billion without Electricy

https://www.iea.org/commentaries/population-without-access-to-electricity-falls-below-1-billion

But there is progress, according to the IEA, The Paris-based International Energy Agency, for the first time ever, the total number of people without access to electriciy fell below 1 billion, I guess that's great, but 1 billion??

And, am sure you can guess where these that still don't have it are - 67% of the developing world, mostly rural areas, again mind boggling.

So, if you are thinking about getting into the energy generating field and what to konw what are the in-demand areas, this will tell you whwere thery are. We really need smart and motivated people to lower that one billion down to zero as quickly as possible.

Two Great Data Sets

We'll use two great data sets, one the World Resources Institute - it is a research organization focusing on different critical issues like climate, energy, food, pollution. We'll use their Global Power Plant Database which lists some 30,000 power plants from 164 countries broken down by types - Hydro, Solar, Wind, Gas, Coal, Oil and more.

WE'll look at power per country and also power per captia. For that we'll join a second great data set from United Nations Department of Economic and Social Affairs which gives current population estimates per country and even extrapolites those number well into the future, some 80 years out - though we won't need this here.

Countries With The Lowest Access To Electricity https://www.worldatlas.com/articles/countries-with-the-lowest-access-to-electricity.html

  • Tanzania (15.3% of population) ...
  • Niger (14.4% of population) ...
  • Sierra Leone (14.2% of population) ...
  • Burkina Faso (13.1% of population) ...
  • Central African Republic (10.8% of population) ...
  • Liberia (9.8% of population) ...
  • Malawi (9.8% of population) ...
  • Burundi (6.5% of population)

https://en.wikipedia.org/wiki/Solar_power_in_Africa

Problems with solar, skeptism and lack of grid to funnel all this power...

Let's see who has energy and who doesn't.

Global Power Plant Database - World Resources Institute

A comprehensive, global, open source database of power plants

http://datasets.wri.org/dataset/globalpowerplantdatabase

Logan Byers - logan.byers@wri.org

World Population Prospects 2019 - United Nations Department of Economic and Social Affairs

Population Dynamics

Total population by sex, annually from 1950 to 2100.

https://population.un.org/wpp/Download/Standard/CSV/

In [148]:
import pandas as pd
global_power_plant_database = pd.read_csv('global_power_plant_database.csv')
In [163]:
global_power_plant_database.shape
Out[163]:
(29910, 24)
In [164]:
global_power_plant_database.head()
Out[164]:
country country_long name gppd_idnr capacity_mw latitude longitude primary_fuel other_fuel1 other_fuel2 ... url geolocation_source wepp_id year_of_capacity_data generation_gwh_2013 generation_gwh_2014 generation_gwh_2015 generation_gwh_2016 generation_gwh_2017 estimated_generation_gwh
0 AFG Afghanistan Kajaki Hydroelectric Power Plant Afghanistan GEODB0040538 33.00 32.3220 65.1190 Hydro NaN NaN ... http://globalenergyobservatory.org GEODB 1009793 2017.0 NaN NaN NaN NaN NaN NaN
1 AFG Afghanistan Mahipar Hydroelectric Power Plant Afghanistan GEODB0040541 66.00 34.5560 69.4787 Hydro NaN NaN ... http://globalenergyobservatory.org GEODB 1009795 2017.0 NaN NaN NaN NaN NaN NaN
2 AFG Afghanistan Naghlu Dam Hydroelectric Power Plant Afghanistan GEODB0040534 100.00 34.6410 69.7170 Hydro NaN NaN ... http://globalenergyobservatory.org GEODB 1009797 2017.0 NaN NaN NaN NaN NaN NaN
3 AFG Afghanistan Nangarhar (Darunta) Hydroelectric Power Plant ... GEODB0040536 11.55 34.4847 70.3633 Hydro NaN NaN ... http://globalenergyobservatory.org GEODB 1009787 2017.0 NaN NaN NaN NaN NaN NaN
4 AFG Afghanistan Northwest Kabul Power Plant Afghanistan GEODB0040540 42.00 34.5638 69.1134 Gas NaN NaN ... http://globalenergyobservatory.org GEODB NaN 2017.0 NaN NaN NaN NaN NaN NaN

5 rows × 24 columns

If you are going to work on aggregates, make sure you are not working on duplicated fields

In [165]:
print(len(global_power_plant_database['gppd_idnr']))
print(len(set(global_power_plant_database['gppd_idnr'])))
29910
29910
In [166]:
# how much capacity per country?
# Maximum Power Output - megawatts (MW)
power_per_country = global_power_plant_database.groupby('country_long')['capacity_mw'].sum().reset_index()
power_per_country = power_per_country.sort_values('capacity_mw', ascending=False)
power_per_country.head(20)
Out[166]:
country_long capacity_mw
30 China 1.363283e+06
155 United States of America 1.193616e+06
67 India 2.886941e+05
124 Russia 2.275556e+05
75 Japan 2.116677e+05
19 Brazil 1.457885e+05
26 Canada 1.434552e+05
52 France 1.092853e+05
57 Germany 1.091628e+05
134 South Korea 9.940678e+04
154 United Kingdom 8.741818e+04
126 Saudi Arabia 8.421855e+04
135 Spain 7.616657e+04
73 Italy 7.157396e+04
7 Australia 6.573117e+04
69 Iran 6.128810e+04
97 Mexico 5.970050e+04
149 Turkey 5.078610e+04
133 South Africa 5.017230e+04
68 Indonesia 4.875142e+04
In [167]:
power_per_country.tail(20)
Out[167]:
country_long capacity_mw
146 Togo 195.114
125 Rwanda 193.680
22 Burkina Faso 176.406
109 Niger 160.200
46 Equatorial Guinea 150.500
63 Guyana 136.300
47 Eritrea 133.190
129 Sierra Leone 126.000
41 Djibouti 107.332
138 Swaziland 94.800
27 Cape Verde 87.805
86 Liberia 73.300
85 Lesotho 72.000
55 Gambia 70.400
14 Benin 60.500
23 Burundi 49.570
28 Central African Republic 33.664
160 Western Sahara 23.400
62 Guinea-Bissau 18.237
4 Antarctica 7.600
In [168]:
# map the ip addresses
import matplotlib
import numpy as np
import matplotlib.pyplot as plt


fig, ax = plt.subplots(figsize=(16, 8))

ax.set_facecolor('xkcd:black')

plt.plot(global_power_plant_database['longitude'], 
         global_power_plant_database['latitude'],
         linestyle='none', marker='.', color='white')

plt.suptitle('World Power Plants')
plt.xlabel('Longitude')
plt.ylabel('Latitude')
plt.grid()
plt.show()

Hydropower is considered a renewable energy

https://www.energy.gov/eere/water/how-hydropower-works

"Hydropower uses a fuel—water—that is not reduced or used up in the process. Because the water cycle is an endless, constantly recharging system, hydropower is considered a renewable energy. When flowing water is captured and turned into electricity, it is called hydroelectric power or hydropower."

In [169]:
global_power_plant_database['primary_fuel'].value_counts()
Out[169]:
Hydro             7155
Solar             5929
Wind              5188
Gas               3922
Coal              2390
Oil               2290
Biomass           1396
Waste             1087
Nuclear            198
Geothermal         189
Storage             58
Other               44
Cogeneration        41
Petcoke             13
Wave and Tidal      10
Name: primary_fuel, dtype: int64
In [171]:
# just one type of power please
tmp_power_df = \
global_power_plant_database[global_power_plant_database['primary_fuel']=='Solar']

fig, ax = plt.subplots(figsize=(16, 8))

ax.set_facecolor('xkcd:black')

plt.plot(tmp_power_df['longitude'], 
        tmp_power_df['latitude'],
         linestyle='none', marker='.', color='white')
print('fig size=', 5 * fig.dpi, 'x',  4 * fig.dpi)
 
plt.suptitle('My Visitors')
plt.xlabel('Longitude')
plt.ylabel('Latitude')
plt.grid()
plt.show()
fig size= 360.0 x 288.0
In [172]:
#You can use agg:
data = global_power_plant_database.groupby('country_long').agg({'capacity_mw':'sum' ,
                                                         'latitude':'mean',
                                                        'longitude':'mean'}).reset_index()

data.head()
Out[172]:
country_long capacity_mw latitude longitude
0 Afghanistan 280.55 34.442229 68.896729
1 Albania 1529.00 41.433675 19.832575
2 Algeria 15203.00 35.318438 3.719072
3 Angola 1071.18 -11.017614 14.405043
4 Antarctica 7.60 -77.844900 166.693800
In [173]:
# you will need to pip install Basemap - https://matplotlib.org/basemap/users/installing.html
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
 
# Set the dimension of the figure
my_dpi=96
plt.figure(figsize=(2600/my_dpi, 1800/my_dpi), dpi=my_dpi)

# Make the background map
m=Basemap(llcrnrlon=-180, llcrnrlat=-65,urcrnrlon=180,urcrnrlat=80)
m.drawmapboundary(fill_color='#A6CAE0', linewidth=0)
m.fillcontinents(color='grey', alpha=0.3)
m.drawcoastlines(linewidth=0.1, color="white")
 
# prepare a color for each point depending on the continent.
data['labels_enc'] = pd.factorize(data['country_long'])[0]
 
# Add a point per position
m.scatter(data['longitude'], data['latitude'], 
          s=data['capacity_mw']/100, alpha=0.4, 
          c=data['labels_enc'], cmap="Set1")
 
# copyright and source data info
plt.text( -170, -58,'Total Power Capacity\nBy Country', ha='left', va='bottom', size=9, color='#555555' )
 
plt.title('Total Power Capacity By Country' , fontsize=50) 
Out[173]:
Text(0.5,1,'Total Power Capacity By Country')
In [52]:
# Throw in population by countries
global_pop = pd.read_csv('WPP2019_TotalPopulationBySex.csv')
global_pop = global_pop[['Location', 'PopTotal', 'Time']]
set(global_pop['Time'])
global_pop = global_pop[global_pop['Time'] == 2020]
global_pop = global_pop.drop_duplicates(keep='last')
global_pop = global_pop.sort_values('Location', ascending=True)

global_pop.head()
Out[52]:
Location PopTotal Time
867 Afghanistan 38928.341 2020
1751 Africa 1340598.113 2020
1838 African Group 1338826.591 2020
1989 African Union 1327533.140 2020
2140 African Union: Central Africa 146728.857 2020
In [59]:
# pd.merge(global_pop, data['homecontinent'], on ['Location']
         
common_df = pd.merge(global_pop, data, how='inner', left_on='Location', right_on='country_long')
In [63]:
common_df['ration_power_pop'] = common_df ['capacity_mw']/ common_df['PopTotal']
In [64]:
common_df.sort_values('ration_power_pop', ascending=False)
Out[64]:
Location PopTotal Time country_long capacity_mw latitude longitude labels_enc ration_power_pop
61 Iceland 341.250 2020 Iceland 2.484600e+03 64.540490 -19.483260 66 7.280879
101 Norway 5421.242 2020 Norway 3.255100e+04 62.182753 10.183628 112 6.004344
9 Bahrain 1701.583 2020 Bahrain 7.581300e+03 26.143857 50.619214 10 4.455439
73 Kuwait 4270.563 2020 Kuwait 1.781950e+04 29.109050 48.112462 80 4.172635
24 Canada 37742.157 2020 Canada 1.434552e+05 47.775718 -86.477261 26 3.800927
111 Qatar 2881.060 2020 Qatar 1.054800e+04 25.352400 51.547050 122 3.661152
139 United States of America 331002.647 2020 United States of America 1.193616e+06 38.966632 -94.737049 155 3.606063
137 United Arab Emirates 9890.400 2020 United Arab Emirates 2.960700e+04 24.911637 54.987593 153 2.993509
81 Luxembourg 625.976 2020 Luxembourg 1.681000e+03 49.731850 6.071500 89 2.685406
125 Sweden 10099.270 2020 Sweden 2.641870e+04 61.718073 16.207856 139 2.615902
6 Australia 25499.881 2020 Australia 6.573117e+04 -31.342074 138.735163 7 2.577705
114 Saudi Arabia 34813.867 2020 Saudi Arabia 8.421855e+04 24.841744 44.273860 126 2.419109
118 Singapore 5850.343 2020 Singapore 1.366000e+04 1.321064 103.713879 130 2.334906
46 Finland 5540.718 2020 Finland 1.179676e+04 62.872534 25.699977 51 2.129103
14 Bhutan 771.612 2020 Bhutan 1.482200e+03 27.228940 90.376820 15 1.920914
43 Estonia 1326.539 2020 Estonia 2.541900e+03 59.047629 24.949865 48 1.916189
47 France 65273.512 2020 France 1.092853e+05 44.824290 1.948830 52 1.674268
69 Japan 126476.458 2020 Japan 2.116677e+05 35.997758 136.767174 75 1.673574
66 Israel 8655.541 2020 Israel 1.447280e+04 31.507034 34.879076 72 1.672085
122 Spain 46754.783 2020 Spain 7.616657e+04 40.166252 -3.648776 135 1.629065
65 Ireland 4937.796 2020 Ireland 7.582150e+03 52.996124 -8.144290 71 1.535533
131 Trinidad and Tobago 1399.491 2020 Trinidad and Tobago 2.144600e+03 10.627867 -61.272150 147 1.532414
126 Switzerland 8654.618 2020 Switzerland 1.309780e+04 46.726898 8.289007 140 1.513389
110 Portugal 10196.707 2020 Portugal 1.441376e+04 39.679805 -9.109365 121 1.413570
90 Montenegro 628.062 2020 Montenegro 8.855000e+02 43.092833 19.054800 100 1.409893
54 Greece 10423.056 2020 Greece 1.454955e+04 38.802941 22.840499 59 1.395901
97 New Zealand 4822.233 2020 New Zealand 6.674550e+03 -40.637340 173.987814 107 1.384120
35 Denmark 5792.203 2020 Denmark 7.897200e+03 55.788503 10.593405 40 1.363419
75 Latvia 1886.202 2020 Latvia 2.537000e+03 56.816780 24.535700 83 1.345031
18 Brunei Darussalam 437.483 2020 Brunei Darussalam 5.860000e+02 4.843850 114.762825 20 1.339481
... ... ... ... ... ... ... ... ... ...
105 Papua New Guinea 8947.027 2020 Papua New Guinea 4.069000e+02 -6.183393 146.015200 116 0.045479
72 Kenya 53771.300 2020 Kenya 2.430200e+03 -1.066386 37.262318 78 0.045195
142 Western Sahara 597.330 2020 Western Sahara 2.340000e+01 23.681600 -15.959400 160 0.039174
56 Guinea 13132.792 2020 Guinea 5.104200e+02 10.424256 -13.283611 61 0.038866
42 Eritrea 3546.427 2020 Eritrea 1.331900e+02 15.499850 39.182150 47 0.037556
115 Senegal 16743.930 2020 Senegal 5.904600e+02 14.594717 -15.968767 127 0.035264
143 Yemen 29825.968 2020 Yemen 1.045000e+03 13.975571 44.435257 161 0.035037
44 Ethiopia 114963.583 2020 Ethiopia 4.023900e+03 9.125485 38.166538 49 0.035002
34 Democratic Republic of the Congo 89561.404 2020 Democratic Republic of the Congo 3.057040e+03 -6.324100 20.702860 39 0.034133
77 Lesotho 2142.252 2020 Lesotho 7.200000e+01 -28.780400 28.453500 85 0.033609
23 Cameroon 26545.864 2020 Cameroon 8.815800e+02 5.349188 10.869700 25 0.033210
3 Angola 32866.268 2020 Angola 1.071180e+03 -11.017614 14.405043 3 0.032592
100 Nigeria 206139.587 2020 Nigeria 6.250000e+03 6.652492 5.938975 110 0.030319
50 Gambia 2416.664 2020 Gambia 7.040000e+01 13.461550 -16.660100 55 0.029131
130 Togo 8278.737 2020 Togo 1.951140e+02 6.587967 1.378200 146 0.023568
83 Malawi 19129.955 2020 Malawi 3.691000e+02 -15.714060 34.864820 92 0.019294
95 Nepal 29136.808 2020 Nepal 5.579500e+02 27.836875 84.880975 105 0.019149
135 Uganda 45741.000 2020 Uganda 7.285000e+02 0.242225 31.902125 151 0.015927
117 Sierra Leone 7976.985 2020 Sierra Leone 1.260000e+02 8.443900 -12.421133 129 0.015795
85 Mali 20250.834 2020 Mali 3.115000e+02 12.957500 -9.483375 94 0.015382
113 Rwanda 12952.209 2020 Rwanda 1.936800e+02 -1.942364 29.596727 125 0.014953
78 Liberia 5057.677 2020 Liberia 7.330000e+01 6.430950 -10.730000 86 0.014493
82 Madagascar 27691.019 2020 Madagascar 3.349300e+02 -19.170809 47.371036 91 0.012095
57 Guinea-Bissau 1967.998 2020 Guinea-Bissau 1.823700e+01 11.883300 -15.633300 62 0.009267
20 Burkina Faso 20903.278 2020 Burkina Faso 1.764060e+02 11.697120 -1.439080 22 0.008439
0 Afghanistan 38928.341 2020 Afghanistan 2.805500e+02 34.442229 68.896729 0 0.007207
25 Central African Republic 4829.764 2020 Central African Republic 3.366400e+01 4.632200 18.325400 28 0.006970
99 Niger 24206.636 2020 Niger 1.602000e+02 16.027467 5.913067 109 0.006618
13 Benin 12123.198 2020 Benin 6.050000e+01 6.388450 2.368050 14 0.004990
21 Burundi 11890.781 2020 Burundi 4.957000e+01 -2.968400 29.344600 23 0.004169

146 rows × 9 columns

Map out power to inhabitant per country

In [174]:
#You can use agg:
data = common_df.groupby('country_long').agg({'ration_power_pop':'sum' ,
                                                         'latitude':'mean',
                                                        'longitude':'mean'}).reset_index()

data.head()
Out[174]:
country_long ration_power_pop latitude longitude
0 Afghanistan 0.007207 34.442229 68.896729
1 Albania 0.531309 41.433675 19.832575
2 Algeria 0.346696 35.318438 3.719072
3 Angola 0.032592 -11.017614 14.405043
4 Argentina 0.716894 -32.793723 -63.318758
In [177]:
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
 
# Set the dimension of the figure
my_dpi=96
plt.figure(figsize=(2600/my_dpi, 1800/my_dpi), dpi=my_dpi)

# Make the background map
m=Basemap(llcrnrlon=-180, llcrnrlat=-65,urcrnrlon=180,urcrnrlat=80)
m.drawmapboundary(fill_color='#A6CAE0', linewidth=0)
m.fillcontinents(color='grey', alpha=0.3)
m.drawcoastlines(linewidth=0.1, color="white")
 
# prepare a color for each point depending on the continent.
data['labels_enc'] = pd.factorize(data['country_long'])[0]
 
# Add a point per position
m.scatter(data['longitude'], data['latitude'], 
          s=data['ration_power_pop']*1000, alpha=0.4, 
          c=data['labels_enc'], cmap="Set1")
 
plt.title('Total Power Capacity By Person', fontsize=50)
plt.text( -170, -58,'Total Power Capacity\nBy Person', ha='left', va='bottom', size=9, color='#555555' )
 
 
Out[177]:
Text(-170,-58,'Total Power Capacity\nBy Person')

Show Notes

(pardon typos and formatting -
these are the notes I use to make the videos)

Let's use two great, frequently curated data sets to map the power plants from around the world by both country and population and identify who still doesn't have access to this essential and life-altering source of energy.