NYPD Crime #19 – Clustering To Explore Neighbourhoods (Part IV – Continued Because Spark Hates Me)

Review I ran into a major dead end in the last post. The problem? Data pre-processing… You don’t often think that data processing would be the activity that prevents you from moving forward eh? As a novice data scientist, you’re so infatuated with the high level objective, the meat of the analysis, the sexy chart […]

Read More NYPD Crime #19 – Clustering To Explore Neighbourhoods (Part IV – Continued Because Spark Hates Me)

NYPD Crime #18 – Clustering To Explore Neighbourhoods (Part III – Continued Because Spark Hates Me)

Review To sum up the last post, our driver’s RAM was essentially the bottleneck and what was causing our Spark application and the underlying JVM to crash. Before, we were using 3 AWS m4.large (8GB RAM) boxes for our master + 2 workers. In this notebook, I’ve spawned a new cluster keeping my workers the […]

Read More NYPD Crime #18 – Clustering To Explore Neighbourhoods (Part III – Continued Because Spark Hates Me)

All-NBA Predict #15 – Exploration of Historical NBA Players (Part VI, PCA to Explore Historical Lineups)

Last time, I explored 9 regions of our PCA bi-plot. I’ll review what I thought the 9 regions represented here: Region 1: All-Star Guards Region 2: All-Star Scorers Region 3: All-Star Big Men Region 4: Great Guards Region 5: Great All-Around / Two-Way Players Region 6: Great Big Men Region 7: Spot Up Shooters Region […]

Read More All-NBA Predict #15 – Exploration of Historical NBA Players (Part VI, PCA to Explore Historical Lineups)

All-NBA Predict #14 – Exploration of Historical NBA Players (Part V, PCA to Cluster Playing Styles)

In our last post, we harnessed the power of PCA even further to look at different ways to break up the PCA bi-plot. With ggbiplot’s “ellipse” argument, we were able to check out where different players lie on the plot. To review, my 3 main questions were: How closely can I map different “types” of […]

Read More All-NBA Predict #14 – Exploration of Historical NBA Players (Part V, PCA to Cluster Playing Styles)