Overview of the TIFIA Program
Analyzing a program without easily downloadable data
I. Introduction
This article provides an introduction to the Transportation Infrastructure Finance and Innovation Act of 1998 (TIFIA) Program. This program is an important source of credit assistance for transportation infrastructure projects in the US and it helps the projects achieve their funding requirements by offering terms that are often “more advantageous terms than in the financial market” [1].
The program is managed by the US Department of Transportation’s Build America Bureau [2] and the TIFIA website [3] provides background information on the program such as the goals and objectives, eligibility requirements, documentation requirements, program fees, etc.
We sought out to analyze the projects and the credit assistance offered by the TIFIA program. The projects are listed on the TIFIA website and for each project, there is a lot of valuable information available on the website. However, this information is not downloadable in a dataset making it difficult to do any kind of analysis. This leaves the analyst with a couple of options (apart from not doing the analysis):
- One option is to take the data from the website and manually add it in a spreadsheet or database. This is not a particularly attractive solution as it can be quite time consuming.
- The second option is to look at the website “page source” and see if the data is organized in a systematic way. If it is then it’s possible to run a “web-scraper” to go through each of the project pages and pick out the information of interest.
Fortunately, the TIFIA project webpages [4] have the data organized in a systematic way, and it was an easy task to write a simple “web-scraper” script to get the data. This script was written in python language and it is provided at the end of this article for the interested analyst (see [5] for more details on web-scraping tools and techniques). Note that most US government websites organize the data displayed in a systematic way and similar web-scrapers can be utilized to generate a dataset for further analysis.
Once the data is gathered and stored in a dataset, the analysis can be done using any number of standard statistical software. For the analysis presented in this article we use the R software platform as it is open-source, free to use and has a very active developer/user community that not only continuously adds to the platform functionality but is also very helpful in answering questions.
II. Review of the TIFIA Program
2.1 Projects by Sector
The TIFIA website provides information on 82 projects (as of Nov. 20, 2020) that the program has considered over the past two decades. A majority of the projects are in the Roads and Bridges sector, followed by Public Transit sector; Figure 1 shows the number of projects by sector.
Over the past two decades, the TIFIA program has provided approx. $29 billion in assistance in direct loans or loan guarantees. Figure 2 shows the assistance provided by sector. Consistent with the number of projects, the Roads and Bridges sector received the highest amount of assistance followed by the Public Transit sector.
2.2 Projects by Status
TIFIA, like other credit programs, has a rigorous application process with multiple stages. The data shows that a majority of the projects have the credit agreement executed while a few of the projects are in the review process (see Figure 3).
2.3 Projects by Instrument Type
The TIFIA program allows for either a direct loan to the project or a loan guarantee. To date most of the projects have received a direct loan (see Figure 4) and only one project has received a guarantee.
2.4 Projects by Year and Sector
The TIFIA program has been active for the past two decades. Figure 5 shows the evolution of the program; in the first few years the amount of projects funded were modest and increasing steadily. The program was quite active in the mid 2010s.
The assistance provided by the program (see Figure 6) reached a peak in year 2014 and has generally been consistent with the number of projects funded. The assistance is also consistent with the number of projects funded by sector.
2.5 TIFIA Assistance at Project Level
So far we have looked at the data at an aggregate level. It is instructive to look at the TIFIA assistance at a project level. Figure 7 plots the assistance provided by sector for individual projects. As is clear from the figure, there is a wide range assistance provided within the sectors with several projects getting less than $100 million and a few projects getting over a billion $s.
III. Closure
In this article, we have provided an overview of the TIFIA program along with some indicators of number of projects funded by sector, timeline and assistance provided. The data used for analysis is not available from the TIFIA website in an easy to download dataset; therefore, we used a simple web-scraper script to get the data.
References
- US Department of Transportation. TIFIA Program Overview. https://www.transportation.gov/buildamerica/financing/tifia/program-overview.
- US Department of Transportation. Build America Bureau. https://www.transportation.gov/buildamerica/.
- US Department of Transportation. TIFIA Credit Program Overview. https://www.transportation.gov/buildamerica/financing/tifia/tifia-credit-program-overview.
- US Department of Transportation. Financed Projects and Applicants. https://www.transportation.gov/buildamerica/projects/financing-search. Data downloaded on 20 Nov 2020.
- Wikipedia. Web Scraping. https://en.wikipedia.org/wiki/Web_scraping.
Web-scraper Code
# PYTHON 3.8
# program to scrape TIFIA project infofrom bs4 import BeautifulSoup
import urllib.request
import pandas as pdraw_link = "https://www.transportation.gov/buildamerica/projects/financing-search?page="
proj_list = list()# first go thru the summary pages to get the links to
# individual project pages.
# loop thru all the pages.
for page_num in range(0,100):
page_link = raw_link + str(page_num)
print(page_link)
# get the page contents
tifia_page = urllib.request.urlopen(page_link).read()
html = BeautifulSoup(tifia_page,"html.parser")
# scrape the page.
# Relevant data is in ARTICLE
# find all the page articles
proj = html.find_all('article', class_='node__content view--item clearfix project__teaser')
print('Number of projects: ',len(proj))
for a_proj in proj:
link = a_proj.find('a')['href']
proj_list.append(link)
if len(proj) < 10:
print('reached last page: ', page_num)
break# now loop thru individual project pages.
all_proj_dict = dict()
for a_proj in proj_list:
proj_link = 'https://www.transportation.gov' + a_proj
print(proj_link)
# get the page contents
proj_page = urllib.request.urlopen(proj_link).read()
proj_html = BeautifulSoup(proj_page,"html.parser")
# scrape the page.
# Relevant data is in ARTICLE
# each important data point is a list (li) item
arts = proj_html.find_all('article')
print('number of articles found',len(arts))
my_dict = dict()
for a_a in arts:
# get the title
a_title = a_a.find_all('h1', class_="node__title")
my_dict['Project Title'] = a_title[0].text.strip('\n')
all_fields = a_a.find_all(class_='field')
for a_field in all_fields:
a_f_l = a_field.find_all('div',class_='field__label')
a_f_i = a_field.find_all('div',class_='field__item')
if len(a_f_l) > 0:
k1 = a_f_l[0].text
v1 = a_f_i[0].text
my_dict[str(k1)] = str(v1)
# store the data in all_proj_dict
all_proj_dict[a_proj] = my_dict# convert the list to a dataframe
v_l = list()
for k,v in all_proj_dict.items():
v_l.append(v)
df = pd.DataFrame(v_l)# OPTIONAL - change column names
# makes life easier in subsequent steps
for a_col in df.columns:
n_col = a_col.lower().replace(' ','_').replace('/','_')
df.rename(columns = {a_col:n_col}, inplace = True)# save the dataframe to a CSV file.
df.to_csv('tifia_data_nov2020.csv',index=False)print('All done!')
