The lubridate package

It’s about time! Lubridate makes working with dates much easier.

We can find how much time has elapsed, add or subtract days, and find seasonal and day of the week averages.

Convert text to a DATE

Get date parts

Install lubridate

First, type or copy and paste install.packages("lubridate") into RStudio.

Clean the dates

Let’s use the mdy() function to turn the start_run_date column into a nicely formatted date.

library(lubridate)

mag_endor <- mutate(mag_endor, date = mdy(start_run_date),
                               year = year(date))

According to the request we received, the Resistance is only interested in observations from the year 2017. So let’s filter the data down to only dates within that year.

mag_endor <- filter(mag_endor, year == "2017")

Time series

ggplot(mag_endor, aes(x = date, y = result)) + 
  geom_line(size = 2, color = "tomato") + 
  geom_point(size = 5, alpha = 0.5)  # alpha changes transparency

Looks like the measurements definitely picked up a signal towards the end of the year. Let’s make a different chart to show when this occurred.

A scrappy ransom

Mr. Baddy Plutt is demanding 10,000 items of scrap for Finn. Sounds expensive, but lucky for us he didn’t clarify the exact items. Let’s find the scrap that weighs the least per shipment and try to make this transaction as light as possible.

Take a look at our NEW scrap data and see if we have the weight of all the items.

# What unit types are in the data?
unique(scrap$units)

## [1] "Cubic Yards" "Items"       "Tons"

Or return results as a data frame

distinct(scrap, units)

## # A tibble: 3 x 1
##   units      
##   <chr>      
## 1 Cubic Yards
## 2 Items      
## 3 Tons

Hmmm…. So how much does a cubic yard of Hull Panel weigh?

A lot? 32? Maybe…

I think we’re going to need some more data.

Weight conversion

It took a while, but with a few droid bribes BB8 was able to track down a Weight conversion table from his old droid buddies. Our current data shows the total cubic yards for some scrap shipments, but not how much the shipment weighs.

# The data's URL
convert_url <- "https://rtrain.netlify.com/data/conversion_table.csv"

# Read the conversion data
convert <- read_csv(convert_url)

head(convert, 3)

## # A tibble: 3 x 3
##   item               units       pounds_per_unit
##   <chr>              <chr>                 <dbl>
## 1 Bulkhead           Cubic Yards             321
## 2 Hull Panel         Cubic Yards             641
## 3 Repulsorlift array Cubic Yards             467

Join 2 tables

Adding porg names

Remember our porg friends? How rude of us not to share their names. Wups!

Here they are:

Hey now! That’s not very helpful. Who’s who? Let’s join their names to the rest their data.

The result

Back to scrap land

Let’s apply our new left_join() skills to the scrap data.

scrap <- left_join(scrap, convert, 
                   by = c("item" = "item", "units" = "units"))

scrap <- left_join(scrap, convert)

head(scrap, 4)

## # A tibble: 4 x 8
##   receipt_date item  origin destination amount units price_per_pound
##   <chr>        <chr> <chr>  <chr>        <dbl> <chr>           <dbl>
## 1 4/1/2013     Bulk~ Crate~ Raiders       4017 Cubi~           1005.
## 2 4/2/2013     Star~ Outsk~ Trade cara~   1249 Cubi~           1129.
## 3 4/3/2013     Star~ Outsk~ Niima Outp~   4434 Cubi~           1129.
## 4 4/4/2013     Hull~ Crate~ Raiders        286 Cubi~            843.
## # ... with 1 more variable: pounds_per_unit <dbl>

Total pounds per shipment

Let’s mutate()!

Now that we have pounds per unit we can use mutate to calculate the total pounds for each shipment.

Fill in the blank

scrap <- scrap %>% 
         mutate(total_pounds = amount *  _____________ )

scrap <- scrap %>% 
         mutate(total_pounds = amount * pounds_per_unit)

Total price per shipment

# Calculate the total price for each shipment
scrap <- scrap %>% 
         mutate(credits = ________ * ________ )
         

# Calculate the total price for each shipment
scrap <- scrap %>% 
         mutate(credits = total_pounds  * ________ )

# Calculate the total price for each shipment
scrap <- scrap %>% 
         mutate(credits = total_pounds * price_per_pound)
         

Price per item

Great! Let’s add one last column. We can divide the shipment’s credits by the amount of items to get the price_per_unit.

# Calculate the price per unit
scrap <- scrap %>% 
         mutate(price_per_unit = credits / amount)

sum()

Use summarize() and sum() to find the total credits of all the scrap.

summarize(scrap, total_credits = sum(credits))

mean()

Use summarize() and mean() to calculate the mean price_per_pound in the scrap report.

summarize(scrap, mean_price = mean(price_per_pound, na.rm = T))

median()

Use summarize to calculate the median price_per_pound in the scrap report.

summarize(scrap, median_price = median(price_per_pound, na.rm = T))

max()

Use summarize to calculate the maximum price per pound any scrapper got for their scrap.

summarize(scrap, max_price = max(price_per_pound, na.rm = T))

min()

Use summarize to calculate the minimum price per pound any scrapper got for their scrap.

summarize(scrap, min_price = min(price_per_pound, na.rm = T))

nth()

Use summarize() and nth(Origin, 12) to find the name of the Origin City that had the 12th highest scrapper haul.

Hint: Use arrange() first.

arrange(scrap, desc(credits)) %>% summarize(price_12 = nth(origin, 12))

quantile()

Quantiles are great for finding the upper or lower range of a column. Use the quantile() function to find the the 5th and 95th quantile of the prices.

summarize(scrap, 
          price_5th_pctile  = quantile(price_per_pound, 0.05, na.rm = T),
          price_95th_pctile = quantile(price_per_pound, 0.95))

Hint: Add na.rm = T to quantile().

summary <- scrap %>% 
             summarize(max_credits      =  __________ ,
                       mean_credits     =  __________ ,
                       min_pounds       =  __________  )

n()

n() stands for count. It has the smallest function name I know of, but is super useful.

Use summarize and n() to count the number of reported scrap records going to Niima outpost.

Hint: Use filter() first.

niima_scrap <- filter(scrap, destination == "Niima Outpost") 

niima_scrap <- summarize(niima_scrap, scrap_records = n())

Ok. That was fun. Now let’s do a summary for Cratertown. And then for Blowback Town. And then for Tuanul. And then for…

The junk Capital of Jakku

Which origin city had the most shipments of junk?

Use group_by with the column origin, and then usesummarize to count the number of records at each origin city.

Fill in the blank

scrap_shipments <- group_by(scrap, ______ ) %>% 
                     summarize(shipments =  ______ ) 

scrap_shipments <- group_by(scrap, origin ) %>% 
                     summarize(shipments =  n() ) 

Bargain hunters

Who’s selling goods for cheap?

Use group_by with the column origin, and then usesummarize to find the mean(price_per_unit) at each origin city.

mean_prices <- group_by(scrap, origin) %>% 
                 summarize(mean_price = mean(price_per_unit, na.rm = T)) %>%
                 ungroup()

mean_prices <- group_by(scrap, origin) %>% 
                 summarize(mean_price       = mean(price_per_unit, na.rm = T),
                           mean_price_round = round(mean_price, digits = 2)) %>%  
                 arrange(mean_price_round) %>%
                 ungroup()