Code for Philly's DemHacks 2016

Code for Philly held a hackathon over the weekend under the theme "Hacks for Democracy." Although my original project did not pan out over the weekend there was a lot of good brainstorming and ideas that I would love to carry over through the summer. Plenty of great ideas did coalesce over the weekend that looked pretty interesting.

One of my favorite projects was Jail Jawn.  This project takes some pretty horribly formatted information from Philadelphia's Prison's daily census and displays the data in charts and graphs that can actually be used to look at the data and try to interpret it.  One of the biggest results from this project is it showed how rarely the data is updated, and hopefully will lead to better updates and knowledge from being able to see this data.

Within the last 20 minutes of the event I overhead the group next to me talking about their project, which was exactly the same as something I had looked into a couple of years ago involving voter turnout. Their project, , Elect Me, looks at the easiest possible political positions that you could be elected to.

Elect Me ranks your best possible options based on the following factors:

1.  Does it pay?

Some jobs such as election judge, pay $95 for the one day of the year that you actually have to serve in the postion.

2.  Ballot filing fee.

Many are free to get your name on the ballot.

3. Least number of votes required.

Some only require a single, yeah just ONE, write-in vote.That's it and you can be elected to something!


In 2014 I briefly looked into the elections for Ward Committee Persons and posted here about it (ugh, those were some ugly maps I made). These are elected officials for each party, that are voted within very small geographic boundaries, Ward Subdivisions.

In many of these wards, committee people were elected via write-in in 2014, with a large number winning the election with a TOTAL of only 1 to 3 votes.

Committee people vote for ward leaders, who are in charge of get out the vote campaigns for general elections.  They also receive any street money distributed by the part for such events.  In addition to this role, ward leaders also pick which candidates from their party, such as judges, will go on the ballot.  Sooooo the committee doesn't directly do any of this, but for a single write in vote you might get invited to the big boys table to vote for whoever does.

Committee people might do other stuff. Who knows.

Within the last few minutes of the hackathon I quickly threw some of the elections data that I had together and shared the following maps through CartoDB for the Elect Me project:

Philadelphia Ward Committee Elected Write-Ins 2014

Philadelphia Ward Committee Total Votes 2014

If you want to get involved and help out with any of the projects I mentioned, or have an idea of one of your own show up any Tuesday at one of Code for Philly's meetup events.  Half the people are computer nerds who need ideas, and the other half are normal people with great ideas, that need computer nerds to help them make it happen. Check them out codeforphilly.org/

Mapping Community Assets in Mumbai with Google's Places API

I mentioned earlier that I was working with a team in a design studio focused on the waterfront in Mumbai.  (Our work will be published in print and at resilientwaterfronts.org.) Mumbai is absolutely fascinating and words cannot describe what it is like to be there and see all of the activity taking place.

The island city of Mumbai is the heart of its financial and economic district. Its shape as a narrow peninsula directly impacts how the city has developed. Mumbai is twice as dense as New York City, and five times as dense as Shanghai. I've found it useful to compare Greater Mumbai and the island city to New York's five boroughs and Manhattan Island.

Within the island city, our study area is the old port on the eastern waterfront. A new modern port is currently in operation across the harbor in Navi Mumbai, and the older port on the island city has much shallower waters that can't accommodate large container ships. The focus of our studio is to study the 1,800 acres that comprise the old port that lie within the heart of one of the densest cities in the world underutilized. The government of India has been building a lot of momentum in looking at redeveloping at least 2/3 of this area.  In a city as dense and active as Mumbai, an opportunity like this is a huge one-time chance to shape the future of this city and open up even greater potential for its development.

The base map below, created by the designers in my studio team, highlights the 1,800 acre port on the eastern coast of Mumbai that we are studying:

Before traveling there we did a lot of prep work gathering data on the existing conditions. However we found that Mumbai was a case in which finding GIS data can be hard to come by.

Part of our planning process started with gathering an inventory of the existing community assets. Open streets map (OSM) has a lot of data but I found that it wasn't 100% complete, nor was it specific to the topics we wanted to focus on. I decided to use the Google Places API tool to find the data we needed to help complete a community assets inventory.

With the Google Places API I was able to search a few areas and grab results for high schools, churches/mosques/temples, and hospitals. The API however only limits you to 60 results which is unfortunate. I mitigated this restriction by defining a short radius of about a 1/4 mile around several points along the study area.  In this case we used the train stations for the Harbor Line, which runs down along the eastern coast of Mumbai.

The Google Places API has some documentation Here.  Basically you can sign up for an API key that will allow you to use the tool. Sometimes APIs, like this one area really easy to use, even if you don't have any programming knowledge at all. Basically you enter all the information you want into the search bar and fill in the various variables such as "Lat=" & ""Lon=" and then a key word or topic like "keyword=churches".  You'll get a result thats probably in XML or some other code that you can save to your computer and open in Excel using the paste special command and "Unicode text" or "XML" to format the results.

Here's an example of the url I typed in (no spaces all one long line) for a lat/long of 18.9442, 72.835 to find all the art galleries in an area:

"https://maps.googleapis.com/maps/api/place/nearbysearch/xml?location=18.9442,72.835 &radius=1000 &keyword=%22art%20gallery%28 &key=Secret_API_Key

"

APIs can be tricky and you'd need to know some programming if you wanted to include the information on a website. However if you are like me and you just want to output the data in a csv file or something simple from a series of results it can be pretty simple. You just type in the url, save it as an xml file, and then you can open it in excel.


In this case I only had about 10 points to search, and up to 3 pages of results for each. If I wanted I could have done this manually without too much time or effort.

However if you need to enter in a very long list, lets say 800 addresses, then you would want to run a script in Python or something that could automatically take that list, query all of the results for you (by plugging the address into the searchbar format that the API wants) and then downloading all of the results into a file. This can be a little trickier sometimes, but its definitely not too difficult to learn. If you are intimidated by coding, you can still use APIs by manually entering in a handful of searches.


From there the results could be placed back on a map in GIS using the Lat/Lon coordinates in the results file and here are the results:

We were able to get a decent list of results by using the this API and mapped the results.  We've since combined these results with other data, such as GIS and population density, as well as neighborhood shapefiles that we digitized into GIS from other sources.

Site Selection for Heathcare Enrollment Support (Using ACS Data)

While working with a team in a competition held by the University of Pennsylvania's Fels Institute of Public Policy, we wanted to create a tool that would conduct a site selection analysis to identify a focus area for our submission.  Our overall project goal was to develop a strategy that can analyze the healthcare information added recently in the American Community Survey (ACS) and identify a focus area during the following healthcare enrollment period.  A tailored approach would then be selected within the area to increase enrollment rates and provide support in picking the best plan for each individual/family.  The advantage of developing a tool like this is that it would be cheap to create and implement, and could provide analysis for any town/city/area since it uses ACS data which is standard throughout the country.

We identified 4 factors from the ACS to be used for site selection:

1. Highest total number of households whose incomes were between 138-399% of the federal poverty level.

2. Greatest number of persons within the 25-34  age range as identified on the ACS.

3. High levels of persons employed but without healthcare.

4. High totals of persons whose healthcare is purchased through the public exchanges.


The totals  for each factor were divided into thirds and a score of 1 to 3 was assigned to each factor as are illustrated below:

The geographical area used within these factors are the 2010 U.S. Census tracts for Philadelphia.  However this approach can be applied to census tracts in other cities and regions as well.

To pick the best particular sites within high scoring areas, the score for each census tract was combined with the scores of its neighbors.  Then the total score in a tract was divided by the total number of neighboring tracts to create an average. As a result, a large tract such as the one in the center of Philadelphia that contains the large green swath of Fairmount Park, and neighbors about 20 different census tracts is left with average score comparable to a small tract with only 5 neighbors. The end result of this process will identify the census tracts with a high score that are also surrounded by the best group of other tracts that scored highly as well.

Below are the results of the final site selection analysis:

Based on this site selected we have 2 different approaches:

1.  One approach for a small clustered area.

The southern site reflects this type of result. The best approach for signup and advising support in this area may be to open an on-site enrollment station within a library or other public setting.  The dense compact geography of this site could thereby be suitable for one central site that people can walk to for one-on-one support.

2. And a different approach for a dispersed geographic region.

The area to the north is fairly large and spread out over a wide area.  In this case opening one on-site center may not be the most efficient way to reach our target group. Instead we may look to open a call center (or in an area with high internet usage a website tool with live chat), and mailed materials or flyers that communicate the availability and contact information for our virtual support option.

Starting in 2013 the ACS has added additional questions to its survey to collect data on the availability of internet and computing in households.  A selected approach for either dense clustered sites, or more dispersed area could be tailored even further depending on the results of this additional information.


Modeling Assumptions and Groundrules

There were however a few issues and assumptions used with the ACS data.  The first to note is the margin of error.  The ACS is a statistical survey about a region.  In this case the geography is a census tract.  However since the surveys have just started and do not have a complete collection of data, the 5 year ACS, may only have information from these questions from within the past 2-3 years.  So for example, the count of uninsured persons could be listed as 52 in a tract, but the margin of error may be huge, like +/- 80.

However as the data quality increases over the next few years, this analysis process will become more effective.  Using the data today is also still a good exercise for developing a process and illustrating its usefulness. The goal of this strategy is to use common data that is publicly available from the ACS, that allows for this process to be replicated anywhere within the U.S.

Another assumption used is that the scores were not weighted.  The process could be revised for example, to weigh age and income more greatly in the final score than total persons with public healthcare. Instead in this case all of the factors were held equal.

Finally we are assuming that the various pieces of information overlap within the same groups we are attempting to target.  For example, we are assuming that separate information about high levels of younger persons overlap with the data indicating a high number of employed persons without healthcare.

(Side Note:  The color selections for the maps were chosen using colorbrewer2.org. A great resource for color palette recommendations.)

Remote Sensing Using Multispectral Analysis

I was pretty stoked to be able to learn how to use remote sensing tools in GIS. To determine the growth in urban land cover, I used multispectral imaging and analysis to compare two images of Mombasa, Kenya from 1992 and 2014.

GIS software can identify urban and non-urban land cover, and combine the two images and measure the growth in urban land cover. Over 12 years the city grew by 86% in land over. According to the Kenya census, the population also doubled from about 460,000 people to a million during that same period.

How does Remote Sensing and Multispectral Mapping actually work? Below is a 3 min video from a guy in a turtleneck explaining it:

  

GIS software has the ability to identify different patterns of images on maps, and pick our urban land, vegetation, water and other uses. The tools in Arcmap can conduct both supervised and unsupervised classifications.  When unsupervised, the tool will basically go through and classify all of the various patterns it finds on its own.  Since computers aren't as smart as people on their own as picking out patterns, this can lead to a lot of patterns output and the results might not be very enlightening.

However under a supervised classification you can train GIS by selecting samples of an area that represent the pattern for each type of land cover.  So you can select urban and vegetation, water, or desert for example. Now when you run the tool the software will try to match each area to the closest example that you used to train the model  (In my example below this is what I did.)  The power of this tool is pretty extraordinary if you combine it with machine learning, or other data such as the specific light frequencies available from USGS Landsat data.

USGS satellites  have the ability to separate images into various light bands. This is an incredibly useful tool for making patterns of certain features much more pronounced and easy to train a model to identify. Using both visible light, as well as infrared and heat imagery, you can combine different combinations to more clearly identify differentiate objects. Combining two different bands can filter out the shaded side of a hill, and leave a unique signature of the rocks and plants in the area. A false color image can create stark a contrast that delineates urban and non-urban areas.  It should be noted that using light frequencies allow for you to filter out shadows and other features and define features clearly.

Each pixel has a value and when you assign a false color (Red, Green, or Blue), that value is represented as a shade of that color.  However those number values are real frequencies in the light spectrum across the band selected.  If you knew the exact frequency of light reflected from a particular plant, you could use this process to highlight those specifically from everything else including other types of plants in the image.

Below are several false color images of a few combinations that can be created using different bands. Notice how in the first, urban land is green and different geological features are shades of red.  The image in the bottom left of the graphic shows different ocean depths and the reef clearly.  Each combination of light bands highlights different types of features.

The next images shown are the analysis for the two different time periods. The small images show the satellite image of the bands used (5,4,3) and the large images show the classification of land cover that was completed using image analysis tools in GIS. The area identified in red for 2014 denotes the new urban land cover that grew over that period while the pink areas illustrate the original urban area. 

Each pixel represents a square area 30m by 30m.  The total growth in area therefore can be computed by simply counting the total red (New Urban Growth) and pink (Original Urban Cover) pixels.

Here is a list of all the different band combinations and uses for each.