Recent Graduates Employment Analysis with Descriptive Stats and plots

In [1]:

import pandas
import matplotlib.pyplot as plt
%matplotlib inline

In [3]:

recent_grads = pandas.read_csv("/Users/33Phoebe/Documents/OneDrive/Data Scientist Path/Data Sets/recent-grads.csv")
recent_grads.iloc[0]

Out[3]:

Rank                                        1
Major_code                               2419
Major                   PETROLEUM ENGINEERING
Total                                    2339
Men                                      2057
Women                                     282
Major_category                    Engineering
ShareWomen                           0.120564
Sample_size                                36
Employed                                 1976
Full_time                                1849
Part_time                                 270
Full_time_year_round                     1207
Unemployed                                 37
Unemployment_rate                   0.0183805
Median                                 110000
P25th                                   95000
P75th                                  125000
College_jobs                             1534
Non_college_jobs                          364
Low_wage_jobs                             193
Name: 0, dtype: object

In [4]:

recent_grads.head()

Out[4]:

	Rank	Major_code	Major	Total	Men	Women	Major_category	ShareWomen	Sample_size	Employed	...	Part_time	Full_time_year_round	Unemployed	Unemployment_rate	Median	P25th	P75th	College_jobs	Non_college_jobs	Low_wage_jobs
0	1	2419	PETROLEUM ENGINEERING	2339.0	2057.0	282.0	Engineering	0.120564	36	1976	...	270	1207	37	0.018381	110000	95000	125000	1534	364	193
1	2	2416	MINING AND MINERAL ENGINEERING	756.0	679.0	77.0	Engineering	0.101852	7	640	...	170	388	85	0.117241	75000	55000	90000	350	257	50
2	3	2415	METALLURGICAL ENGINEERING	856.0	725.0	131.0	Engineering	0.153037	3	648	...	133	340	16	0.024096	73000	50000	105000	456	176	0
3	4	2417	NAVAL ARCHITECTURE AND MARINE ENGINEERING	1258.0	1123.0	135.0	Engineering	0.107313	16	758	...	150	692	40	0.050125	70000	43000	80000	529	102	0
4	5	2405	CHEMICAL ENGINEERING	32260.0	21239.0	11021.0	Engineering	0.341631	289	25694	...	5180	16697	1672	0.061098	65000	50000	75000	18314	4440	972

5 rows × 21 columns

this dataset is based on American Community Survey 2010-2012 Public Use Microdata Series, edited and provided by FiveThirtyEight

In [5]:

recent_grads.tail()

Out[5]:

	Rank	Major_code	Major	Total	Men	Women	Major_category	ShareWomen	Sample_size	Employed	...	Part_time	Full_time_year_round	Unemployed	Unemployment_rate	Median	P25th	P75th	College_jobs	Non_college_jobs	Low_wage_jobs
168	169	3609	ZOOLOGY	8409.0	3050.0	5359.0	Biology & Life Science	0.637293	47	6259	...	2190	3602	304	0.046320	26000	20000	39000	2771	2947	743
169	170	5201	EDUCATIONAL PSYCHOLOGY	2854.0	522.0	2332.0	Psychology & Social Work	0.817099	7	2125	...	572	1211	148	0.065112	25000	24000	34000	1488	615	82
170	171	5202	CLINICAL PSYCHOLOGY	2838.0	568.0	2270.0	Psychology & Social Work	0.799859	13	2101	...	648	1293	368	0.149048	25000	25000	40000	986	870	622
171	172	5203	COUNSELING PSYCHOLOGY	4626.0	931.0	3695.0	Psychology & Social Work	0.798746	21	3777	...	965	2738	214	0.053621	23400	19200	26000	2403	1245	308
172	173	3501	LIBRARY SCIENCE	1098.0	134.0	964.0	Education	0.877960	2	742	...	237	410	87	0.104946	22000	20000	22000	288	338	192

5 rows × 21 columns

In [6]:

recent_grads.describe()

Out[6]:

	Rank	Major_code	Total	Men	Women	ShareWomen	Sample_size	Employed	Full_time	Part_time	Full_time_year_round	Unemployed	Unemployment_rate	Median	P25th	P75th	College_jobs	Non_college_jobs	Low_wage_jobs
count	173.000000	173.000000	172.000000	172.000000	172.000000	172.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000	173.000000
mean	87.000000	3879.815029	39370.081395	16723.406977	22646.674419	0.522223	356.080925	31192.763006	26029.306358	8832.398844	19694.427746	2416.329480	0.068191	40151.445087	29501.445087	51494.219653	12322.635838	13284.497110	3859.017341
std	50.084928	1687.753140	63483.491009	28122.433474	41057.330740	0.231205	618.361022	50675.002241	42869.655092	14648.179473	33160.941514	4112.803148	0.030331	11470.181802	9166.005235	14906.279740	21299.868863	23789.655363	6944.998579
min	1.000000	1100.000000	124.000000	119.000000	0.000000	0.000000	2.000000	0.000000	111.000000	0.000000	111.000000	0.000000	0.000000	22000.000000	18500.000000	22000.000000	0.000000	0.000000	0.000000
25%	44.000000	2403.000000	4549.750000	2177.500000	1778.250000	0.336026	39.000000	3608.000000	3154.000000	1030.000000	2453.000000	304.000000	0.050306	33000.000000	24000.000000	42000.000000	1675.000000	1591.000000	340.000000
50%	87.000000	3608.000000	15104.000000	5434.000000	8386.500000	0.534024	130.000000	11797.000000	10048.000000	3299.000000	7413.000000	893.000000	0.067961	36000.000000	27000.000000	47000.000000	4390.000000	4595.000000	1231.000000
75%	130.000000	5503.000000	38909.750000	14631.000000	22553.750000	0.703299	338.000000	31433.000000	25147.000000	9948.000000	16891.000000	2393.000000	0.087557	45000.000000	33000.000000	60000.000000	14444.000000	11783.000000	3466.000000
max	173.000000	6403.000000	393735.000000	173809.000000	307087.000000	0.968954	4212.000000	307933.000000	251540.000000	115172.000000	199897.000000	28169.000000	0.177226	110000.000000	95000.000000	125000.000000	151643.000000	148395.000000	48207.000000

In [7]:

raw_data_counts = len(recent_grads)
recent_grads.shape

Out[7]:

(173, 21)

In [9]:

recent_grads = recent_grads.dropna()
cleaned_data_count = len(recent_grads)
cleaned_data_count

Out[9]:

In [10]:

cols = ["Sample_size", "Median", "Unemployment_rate", "Full_time", "ShareWomen", "Men", "Women"]
recent_grads.plot(x = cols[0], y= cols[1], kind = 'scatter', title = 'Sample size Vs. Median')
recent_grads.plot(x = cols[0], y= cols[2], kind = 'scatter', title = 'Sample size Vs. Unemployment_rate')
recent_grads.plot(x = cols[3], y= cols[1], kind = 'scatter', title = 'Full time Vs. Median')
recent_grads.plot(x = cols[4], y= cols[2], kind = 'scatter', title = 'ShareWomen Vs. Unemployment rate')
recent_grads.plot(x = cols[5], y= cols[1], kind = 'scatter', title = 'Men Vs. Median')
recent_grads.plot(x = cols[6], y= cols[1], kind = 'scatter', title = 'Women Vs. Median')

Out[10]:

<matplotlib.axes._subplots.AxesSubplot at 0x115796860>

Finding:¶

No clear relationship between the popularity of the major to its median income
No clear relationship between the women partaking in a major to its median income
No clear relationship between the number of full_time employees to the median salary
However, comparing women/men Vs. Median, it appears the average of men median income is higher than that of women

In [11]:

ax = recent_grads[cols[0]].hist(bins = 25, range =(0, 400))
ax.set_title("sample size")

Out[11]:

Text(0.5,1,'sample size')

Exploring the histogram of the sample size shows most of the majors have less than 400 people responded, while 17.5% of the majors didn't respond at all.

In [12]:

ax = recent_grads[cols[1]].hist(bins = 25, range =(20000, 80000))
ax.set_title("Median Salary")

Out[12]:

Text(0.5,1,'Median Salary')

Graduates of most majors have a median salary of 30,000~40,000

In [13]:

col = ["Employed", "Full_time", "ShareWomen", "Unemployment_rate", "Men", "Women"]
ax = recent_grads[col[0]].hist(bins = 25, range =(0, 200000))
ax.set_title("Employed")
#Employed concentrates around 0~50000, showing most of the majors are fairly new?

Out[13]:

Text(0.5,1,'Employed')

In [14]:

ax = recent_grads[col[1]].hist(bins = 25, range = (0, 175000))
ax.set_title("Full Time")

Out[14]:

Text(0.5,1,'Full Time')

In [15]:

ax = recent_grads[col[2]].hist(bins = 50)
ax.set_title("ShareWomen")
#Except some, most majors have a fair quantity of women students

Out[15]:

Text(0.5,1,'ShareWomen')

In [16]:

ax = recent_grads[col[3]].hist(bins = 25)
ax.set_title("Unemployment Rate")
#Most Majors have around 6%~8% unemployment rate

Out[16]:

Text(0.5,1,'Unemployment Rate')

In [17]:

ax = recent_grads[col[4]].hist(bins = 25, range = (0, 40000))
ax.set_title("Men")

Out[17]:

Text(0.5,1,'Men')

In [18]:

ax = recent_grads[col[5]].hist(bins = 25, range = (0, 50000))
ax.set_title("Women")

Out[18]:

Text(0.5,1,'Women')

Finding¶

Judging by the sharewomen graph, the majors with predominantly male and female are about the same amount
Most common median salary is 30,000~40,000

In [20]:

from pandas.tools.plotting import scatter_matrix

In [21]:

scatter_matrix(recent_grads[["Sample_size", "Median"]], figsize = (8, 8))

/Applications/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:1: FutureWarning: 'pandas.tools.plotting.scatter_matrix' is deprecated, import 'pandas.plotting.scatter_matrix' instead.
  """Entry point for launching an IPython kernel.

Out[21]:

array([[<matplotlib.axes._subplots.AxesSubplot object at 0x11572e2e8>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x115655278>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x1158ee278>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x115ae2278>]], dtype=object)

In [24]:

scatter_matrix(recent_grads[["Sample_size", "Median", "Unemployment_rate"]], figsize = (9, 9))

/Applications/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:1: FutureWarning: 'pandas.tools.plotting.scatter_matrix' is deprecated, import 'pandas.plotting.scatter_matrix' instead.
  """Entry point for launching an IPython kernel.

Out[24]:

array([[<matplotlib.axes._subplots.AxesSubplot object at 0x1161a1630>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x1168d9400>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x116914400>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x11694c470>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x116972ef0>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x116972f28>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x1169f04a8>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x116a294a8>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x116a63400>]], dtype=object)

In [25]:

recent_grads[:5]["Women"].plot(kind = "bar")

Out[25]:

<matplotlib.axes._subplots.AxesSubplot at 0x115806be0>

In [26]:

recent_grads[:5].plot.bar(x = "Major", y = "Women")

Out[26]:

<matplotlib.axes._subplots.AxesSubplot at 0x116d6e7f0>

In [32]:

sorted_rg = recent_grads.sort_values("Median")
sorted_rg.head()
sorted_rg.tail()

Out[32]:

	Rank	Major_code	Major	Total	Men	Women	Major_category	ShareWomen	Sample_size	Employed	...	Part_time	Full_time_year_round	Unemployed	Unemployment_rate	Median	P25th	P75th	College_jobs	Non_college_jobs	Low_wage_jobs
4	5	2405	CHEMICAL ENGINEERING	32260.0	21239.0	11021.0	Engineering	0.341631	289	25694	...	5180	16697	1672	0.061098	65000	50000	75000	18314	4440	972
3	4	2417	NAVAL ARCHITECTURE AND MARINE ENGINEERING	1258.0	1123.0	135.0	Engineering	0.107313	16	758	...	150	692	40	0.050125	70000	43000	80000	529	102	0
2	3	2415	METALLURGICAL ENGINEERING	856.0	725.0	131.0	Engineering	0.153037	3	648	...	133	340	16	0.024096	73000	50000	105000	456	176	0
1	2	2416	MINING AND MINERAL ENGINEERING	756.0	679.0	77.0	Engineering	0.101852	7	640	...	170	388	85	0.117241	75000	55000	90000	350	257	50
0	1	2419	PETROLEUM ENGINEERING	2339.0	2057.0	282.0	Engineering	0.120564	36	1976	...	270	1207	37	0.018381	110000	95000	125000	1534	364	193

5 rows × 21 columns

In [31]:

sorted_rg[:10].plot.bar("Major", "ShareWomen")

Out[31]:

<matplotlib.axes._subplots.AxesSubplot at 0x116ea1550>

The top 10 lowest income majors have mostly women students, with 3 majors have 90% women and the major with fewest women in this ranking still has more than 60% women.

In [39]:

sorted_rg[len(sorted_rg)-10:len(sorted_rg)].plot.barh("Major", "ShareWomen")

Out[39]:

<matplotlib.axes._subplots.AxesSubplot at 0x117350c88>

In comparison, the top 10 highest paid majors have very low female:male ratio, on average around 15% with some less than 10%

In [40]:

sorted_rg[:10].plot.barh("Major", "Unemployment_rate")
sorted_rg[len(sorted_rg)-10:len(sorted_rg)].plot.barh("Major", "Unemployment_rate")

Out[40]:

<matplotlib.axes._subplots.AxesSubplot at 0x117634320>

These female dominated majors are of a roughly higher unemployment rate than the male dominate majors.

social