Setting up Python in Windows 10

Installing Python under Windows 10 is fairly easy as long as you set up your system environment correctly. Below is my quick guide, which follows similar how-to’s I’ve written for installing Python under Windows 7 and under Windows 8.1.

Ready? Here’s your quick guide:

Set up Python on Windows 10

1. Visit the official Python download page and grab the Windows installer for the latest version of Python 3. One note:

  • Python is available in two versions — Python 2 and Python 3. For beginners, that may be confusing. In short, Python 3 is the current and future state of the language; Python 2 is a legacy version that still has a large base of users. Python 2 will reach its end of life in January 2020 and will only get bug fixes till then.

2. Right-click on the installer and select “Run as Administrator.” Click “Yes” when Windows asks if you want the program to make changes to your computer.

3. The next dialog asks whether you want to “Install Now” or “Customize Installation.” You want to “Customize Installation,” so click that.

4. On the next screen, check all boxes under “Optional Features.” Click next.
Continue…

Analyzing Shapefile Data with PostgreSQL

This is one in a series of posts adapted from material in the book Practical SQL.

Spend some time digging into geographic information systems (GIS) and soon enough you’ll encounter a shapefile. It’s a GIS file type developed by mapping software firm Esri for use in its popular ArcGIS platform. A shapefile contains the geometric information to describe a shape—a river, road, lake, or town boundary, for example—plus metadata about the shape, such as its name.

Because the shapefile has become a de facto standard for publishing GIS data, other applications and software libraries use shapefiles too, such as the open source QGIS.

While researching GIS topics for a chapter in my book, Practical SQL, I learned that it’s easy to import a shapefile into a PostGIS-enabled PostgreSQL database. The information that describes each shape is stored in a column of data type geometry, and so you can run spatial queries to calculate area, distances, intersections of objects, and more.

Here’s a quick exercise, adapted from the book. Continue…

‘Practical SQL’ Book in Early Release

My first book, Practical SQL: A Beginner’s Guide to Storytelling with Data, is out in early release from No Starch Press starting today! If you pre-order from No Starch, you can download the Introduction and first four chapters now. You’ll get additional chapters regularly until the final version comes out in February 2018.

Practical SQL is for people who encounter data in their everyday lives and want to know how to analyze or transform it. The book covers real-world data and scenarios, from analyzing U.S. Census demographics to the duration of taxi rides in New York City. I’ve aimed the exercises at beginning SQL coders, and all the code and data can be downloaded via No Starch’s site.

That database you’ll use is the free, open-source PostgreSQL, along with the pgAdmin 4 graphical user interface. We cover all the basics you’ll find in standard ANSI SQL along with PostgreSQL-specific features such as full text search and GIS.

More to come as additional chapters hit early release!

NoVa-Py Talk: Building a Python Package

One of the most popular uses of the API for DocumentCloud, the document research/publishing platform where I work, is to bulk-upload hundreds or thousands of documents. People usually hack their own code together to do this, sometimes using the Python or Ruby wrappers for the API.

After talking with users and hearing their thoughts about the workflow — a desire to have a record of each file’s URL once uploaded, for example — I saw an opportunity to add some luxury to the process. A couple of months, a lot of research, and a few bruises later, I had my first Python package: pneumatic.

pneumatic does a few things to make life easier. It grabs information about each uploaded file and saves it in a SQLite database, which you can dump to csv. It uses Python’s multiprocessing module to try to add some speed (recognizing that this is a network-bound task). And it scans all subfolders for files, which is handy when you obtain a collection of files organized that way.

Learning about Python packaging was as much a part of the project as creating the library itself. The folks at the Northern Virginia Python Users Group were kind enough to invite me to share what I learned recently. Click through the title card to view the slides.

BaPP

 

Today’s weather in my inbox, via Python

In the category of “potentially useful but mostly just a learning exercise,” here’s a Python script that emails me the local weather report twice a day. I loaded it on a Raspberry Pi my family gave me as a gift last year, set up a cron task, and now each day when I wake up I have a forecast waiting in my inbox. Makes me feel special!

The script — compatible with Python 3.6 and Python 2.7 — uses the awesome Requests library to fetch two endpoints from the Weather Underground API. One provides a forecast, and the other offers a summary of yesterday’s weather. For emailing, it uses the standard Python smtplib.

The code’s available on Github, so fork it and make it your own. You’ll need to have the Requests and simplejson libraries installed. Contributions are welcome!

Here’s a quick overview on how to set it up:

First, you’ll need to sign up for a Weather Underground API key. The free developer level has more than enough calls per day for this app, so choose that unless you plan to obsess about the weather in an oversized manner.

The API key and your email parameters go into a settings.py file:

mail_settings = {
    'address': 'anyone@example.com',
    'pw': 'your-email-password',
    'smtp': 'post.example.com',
    'from': 'Mr. Weather Robot'
}

send_to_addresses = ['someone@example.com', 'someone_else@example.com']

api_key = 'your-wunderground-api-key'

Then, here’s the wx-mail.py file:

import datetime
import smtplib
import requests
import simplejson as json
from email.mime.text import MIMEText
from local_settings import mail_settings, send_to_addresses, api_key


def fetch_forecast(api_key, request_type):
    mail_url = 'http://api.wunderground.com/api/' + api_key + '/' +\
               request_type + '/forecast/q/VA/Leesburg.json'
    r = requests.get(mail_url)
    j = json.loads(r.text)
    return j


def build_html(forecast_json, yesterday_json):
    # build some HTML snippets to open and close this email
    html_open = """\
    <html>
      <head></head>
      <body>
    """
    html_close = """\
      </body>
    </html>
    """

    # let's now build the HTML body contents
    wxdate = forecast_json['forecast']['txt_forecast']['date']
    mail_text = '<h3>Hello, DeBarros family!</h3><p>Here is the ' +\
                'Leesburg, Va., weather forecast as of ' + wxdate + '</p>'
    forecast_length = len(forecast_json['forecast']['txt_forecast']['forecastday']) - 1

    # looping through the JSON object
    for i in range(0, forecast_length):
        cast = '<p><b>' +\
            forecast_json['forecast']['txt_forecast']['forecastday'][i]['title'] +\
            '</b>: ' +\
            forecast_json['forecast']['txt_forecast']['forecastday'][i]['fcttext'] +\
            '</p>'
        mail_text += cast

    # Now, for yesterday's weather summary ...
    # We'll pull the date and some weather data from the summary API endpoint
    summary_date = yesterday_json['history']['dailysummary'][0]['date']['pretty']

    high_low_temp = yesterday_json['history']['dailysummary'][0]['maxtempi'] +\
        ' / ' +\
        yesterday_json['history']['dailysummary'][0]['mintempi'] +\
        ' degrees Fahrenheit'

    max_min_humid = yesterday_json['history']['dailysummary'][0]['maxhumidity'] +\
        '% / ' +\
        yesterday_json['history']['dailysummary'][0]['minhumidity'] + '%'

    precipitation = yesterday_json['history']['dailysummary'][0]['precipi'] +\
        ' inches'

    max_wind_speed = yesterday_json['history']['dailysummary'][0]['maxwspdi'] +\
        ' mph'

    yesterday_html = """\
    <h3>Here's yesterday's weather summary:</h3>
    <p><b>High/low temperature: </b>""" + high_low_temp + '</p>' +\
    '<p><b>Max/min humidity: </b>' + max_min_humid + '</p>' +\
    '<p><b>Precipitation: </b>' + precipitation + '</p>' +\
    '<p><b>Maximum wind speed: </b>' + max_wind_speed + '</p>'

    # put it all together
    html_body = html_open + mail_text + yesterday_html + html_close
    return html_body


def send_email(mail_text):
    # Set the current time and add that to the message subject
    cur_date = datetime.date.today().strftime("%B") +\
        ' ' + datetime.date.today().strftime("%d") +\
        ', ' + datetime.date.today().strftime("%Y")
    subject = 'Family forecast for ' + cur_date

    # Set up the message subject, etc. Then send it.
    COMMASPACE = ', '

    msg = MIMEText(mail_text, 'html')
    msg['Subject'] = subject
    msg['From'] = mail_settings['from']
    msg['To'] = COMMASPACE.join(send_to_addresses)

    server = smtplib.SMTP(mail_settings['smtp'], 25)
    server.login(mail_settings['address'], mail_settings['pw'])
    server.set_debuglevel(1)
    server.sendmail(mail_settings['address'], send_to_addresses,
                    msg.as_string())
    server.quit()


if __name__ == "__main__":
    forecast_json = fetch_forecast(api_key, 'forecast')
    yesterday_json = fetch_forecast(api_key, 'yesterday')
    mail_text = build_html(forecast_json, yesterday_json)
    send_email(mail_text)

The code’s straightforward, but a few things to note:

  • The Python standard smtplib provides all you need for sending the email. Check the official docs for examples.
  • I’ve gotten into the habit of using the simplejson library for wrangling API response objects, but the standard Python json library works just as well.

Have fun, and may all your coding days be sunny and warm.