Author Archives: aimeerosato

“Edible” Tree Species Mapped in NYC

Interest in Agriculture

I have always been interested in agriculture since my humble beginnings as a home gardener in Vermont.  In college, I had a group of friends who were extremely passionate about urban harvesting.  They were concerned about the amount of edible food they saw go to waste around their small city and teamed up to do something about it on a volunteer basis.  When I had the opportunity to dive into the NYC 2015 Tree data, I started thinking about what kind of edible tree species are within the five boroughs.  Also, I often have seen Mulberry trees make a mess of the sidewalk near my house, which made me wonder where all of the Mulberry trees are located specifically.

20160630_170358

Mulberry mess on a street in my neighborhood.  Honestly, it doesn’t look like much because the rain earlier this week washed it away, but it can look like the sidewalk is full of purple jam!

NYC Parks’ TreesCount 2015!

If you read my last post, by now you know all about NYC Open Data and the Park’s Department TreesCount 2015 data.  Basically, the Parks department counts all of the “NYC street trees” every ten years to keep tabs on how the urban forest is doing.   Street trees are all of the trees you see in the NYC sidewalks, and they do not include trees that are in public parks.

When first given a chance to analyze this data, of course my mind started asking questions about how many NYC Street Trees are some kind of edible species. I decided to focus my visualization on that.  But there’s a huge caveat!  At the aforementioned event, there were many Parks Department representatives to support, and from one of them I learned that the Parks Department chooses a lot of hybrid species for their street trees.  This means that although in the list of trees in NYC there are many edibles, the actual trees that are planted may not be fruit bearing because that would make a mess of the sidewalks. (The Mulberry trees I mentioned earlier ended up being on private property that borders the sidewalk.)  Also, since I don’t have more specific information about the trees beyond their common and scientific names, please proceed with caution if you decide to check the trees in this visualization.  The following is the process I took to create this visualization.

Consulting an Expert

My first task was to identify all of the tree species that I wanted visualize from the data.  While I was able to identify some species that I knew had some kind of edible, I consulted with my good friend, Samantha Anderson, who happens to be a landscape architect and general expert in these sorts of things.  With her help, we created a list of all the species that could possibly have something edible, most of them being fruit or nut bearing trees.     In total I ended up with 13 species of NYC Trees to include in the visualization.  (Thank you Sam!  Note: final list did not include sugar maple even though the sap is edible.  Can’t imagine NYC Parks is going to tap some trees anytime soon!)

Visualizing Data with CartoDB

Issue #1: Edible Species Not the Most Common

I used CartoDB for visualizing the trees of interest on a map of NYC.  First, I had to import the TreesCount 2015 Street Tree Data off of the NYC Open Data Portal.    Then, since I wanted my visualization to focus on the common species names for the NYC trees, I experimented with viewing the different categories of tree species by specifying that I wanted the data sorted by the “spc_common” column.

In this screen shot, you can see that ten types of specific trees that are the most common in the data set are each given their own colors and pins.  My problem was that some of the  species I was interested in were lumped into the gray “Other” category.

tree_category_img

Top categories of trees mapped by category

Solution: Custom CSS

To change which species showed up on the map and to identify which colors I wanted their pins to be, all I had to do was change the CSS style for the species that were being targeted in the CartoCSS editor.  At least that’s what I thought I had to do.

Issue #2: Dataset Too Broad

While some of the pins changed colors, I realized that the map was still showing ALL of the trees in NYC.  I had just changed the styling, but what I really needed to change was the underlying dataset that the map was drawing from to show the plotted trees.

Solution: Custom SQL Query

There may be a more efficient way to do this, but I basically created a custom SQL query that targeted only the tree species I was looking for so that the dataset would only reflect that.

 Final Map

After tweaking the legend and other clean up, I finally had all of my edible tree species in NYC mapped!  Check it out here.  You can see what types of edible tree species live in your neighborhood, but please, sample edibles at your own risk!

TreesCount! Data Jam Recap

20160604_101452

Noel Hidalgo from BetaNYC at the kickoff to the TreesCount! Data Jam on June 4, 2016 – part of the National Day of Civic Hacking

National Day of Civic Hacking and BetaNYC

Last Saturday, June 4, 2016 was the National Day of Civic Hacking across the USA, where people from a variety of backgrounds come together to hack on projects for good.  How cool is that!?  It sponsored by Code for America and a bunch of other organizations and federal agencies.  It’s a hackathon where people use government data available to them to work on different challenges.  This year there were challenges with everything from mapping honey bees in your yard to analyzing data about the spread of Zika.

I am very interested in the intersection of technology and civic engagement, so when I discovered that BetaNYC was a civic group already doing this work in NYC I joined their Meetup Group immediately.  The TreesCount! Data Jam was the first event I participated in, and I look forward to going to more!

TreesCount! Data Jam

BetaNYC partnered with the NYC Parks department to create the TreesCount! Data Jam Event on the National Day of Civic Hacking.  They unveiled the TreesCount! 2015 dataset of all the street trees counted within the five boroughs of NYC, and people participated in hacking on 5 specific challenges you can read about here.  Data was also collected on street trees in 1995 and 2005, though the 2015 data had more specific geospatial data because volunteers used TreeKits to map.  Every time they’ve done the count they’ve realized that there are more and more trees on NYC streets!

20160604_095338

Bar graph from the Parks Department comparing the number of trees per selected boroughs in 1995, 2005, and 2015.

At the event kickoff, members of the parks department intro20160604_162046duced the new dataset and gave us information about how to access it in the NYC Open Data Portal.  In fact, there were NYC Parks department employees hanging around all day to answer questions and provide a deeper understanding of how the Parks Department works.  Manhattan Borough President Gail Brewer also gave a lighthearted speech about the importance of trees in NYC, as well as the complaints her office gets from constituents about trees, and now ivy around the trees leads to rats.

Off to work!

There was over 150 people there from all walks of life.  I met UX designers, project managers, developers like me, data scientists, economists, and even volunteers who collected the data and wanted to see how people could use it. After the kickoff, people drifted towards the challenges they were interested in and began to look at the data and hack the day away!

I chose to participate in a workshop that was going on with Julia Marden from BetaNYC and Annarita Macri from the Parks Department.  It was amazing!  We learned how to interact with the NYC Open Data portal, took a deep dive into the NYC Trees and NYC Blockfaces (shows the completion of trees counted on a block level) datasets, made pivot tables, learned about Geocoding, and also worked on mapping the tree data using CartoDB.  Next week I’ll be blogging about the map I made of edible tree species in the NYC urban forest landscape!

End of Day

20160604_174804

You can see the four teams at their numbered laptops ready to present.  The current team was presenting on their data that mapped trees in bad condition in proximity to places like daycares and senior citizen centers.

I have to say that this was one of the most well-run events I have ever attended.  An example of the event’s efficiency has to be the end-of-day wrap-up where about 20 teams who did the hackathon each at a 2 minute (or was it one minute?) period of time to present their project to everyone.  Each project was based on one of the five challenges presented at kick off, and they were judged by a panel of experts on how well they completed the challenge.  After each presentation, the next team was already on stage ready to hook into the overhead screens and present. The presentations and projects turned out great!

One standout for me was a Twitter robot called Every Tree NYC (@everytreenyc) that interacts with the Tree Data to answer the challenge of increasing public engagement with the tree data.  Basically, it pairs pithy (pith – HA!) statements with images and locations of trees around the five boroughs.  For instance, an image of a dead tree came with the caption, “RIP.”  Other projects were more serious in tone than this one, but my love of a good pun is making a little biased here!

The projects were extremely inspiring.  It was amazing to see what people can create when given data and a few hours to hack on it!  I highly suggest this event, and I know I will surely be participating next year.

20160604_180131

My neighborhood pizza parlor also made an appearance in this project that compared weather station data on streets with trees versus without streets!

Civic Hall and Future Events

Also, did I mention that the event took place at Civic Hall – an amazing community center!  You should definitely check it out!  If you want to become a member you do have to pay a monthly fee, but they host a lot of interesting events.  For instance, next Tuesday, June 7th at 12:30 they hosting an event where they are launching the 311 Call Center Inquiry dataset.

Slugification

When entering a new field, you not only have to learn a new set of skills, but also the industry jargon you might not be familiar with.  Before becoming a software engineer, a ‘slug’ to me was a pest in our garden that my siblings and I would pour salt on and/or run away from.  Now ‘slugs’, and what it means to ‘slugify’ a string, have taken on a whole new meaning.

Slug

A slug is a part of a website’s url, usually the part at the end that comes from the title of the page or section.  On this blog, if you were to click on the “About” section, the url would become https://aimeerosato.com/about-2/.  The slug, “about-2”, comes from the fact that it’s about the “About” page.  By convention, slugs are usually lowercase, have spaces in a phrase replaced by a dash or underscore, and have punctuation marks removed.  Sometimes smaller words are dropped or changed to make the slug more readable.  For example, with the article entitled “The U.S. is still using floppy disks to run its nuclear program” from cnn.com, http://www.cnn.com/2016/05/26/us/pentagon-floppy-disks-nuclear/index.html, you can see the slug for that article became “pentagon-floppy-disks-nuclear.”    Interesting article by the way – I remember those huge floppy disks!

Tools to Slugify

When you are writing a program, you might find the need to turn a string into a slug, i.e. to slugify it.  While you can spend the time writing code that cleans up your string, many people have created tools that are ready for you to use.  For instance, there’s an npm package to help with this and also a lodash addon.  A quick search around the internet leads to many other tools like these to help slugify strings.   Happy slugifying!

 

Event Review and Tips: NYC Uncubed

Last Friday, I attended the NYC Uncubed Spring Event, which combines a job fair with a bunch of neat workshops and speakers (http://nyc.uncubed.com/).  It was hosted at the Met Pavilion on 18th Street in Manhattan, and I had a great time. However, it is not a free event, and I bought my ticket online ahead of time for around $35 bucks after using a discount code I found online.

At the event, registration was quick and efficient.  Every attendee got a lanyard with a huge name tag that was color-coded so you could see their background, for instance, mine was yellow for being in tech.  This made networking with other people at the event easier because you could visual find people with your background and talk to them about their experience.

Pre-Event Tips

  1. Research Companies: I spent a few hours researching the companies that were going to be at the fair ahead of time.  I checked out their missions and open positions and made a short list of the companies I wanted to talk to.
  2. Practice your Elevator Pitch: You will have to answer the dreaded, open-ended question, “Tell me about yourself” multiple times at the event.  It helps if you aren’t saying it for the first time with a cool company you would love to work at!  🙂
  3. Resume: It was helpful to bring hard copies of my updated resume to the event.  Even though some companies encouraged me to apply online, other people I talked to looked at the resume to frame our conversation.

At Event Tips

  1. Have an Open Mind: Sometimes current open positions at companies have changed slightly from when you did your research.  This is a blessing and a curse because you don’t have to waste time applying for jobs you know don’t exist anymore, but…that job you wanted doesn’t exist anymore.  😦
  2. Crowds: There were a LOT of people there.   People literally wait in lines to talk to people from hiring companies.  However, the great thing was that most company booths had a bunch of employees, so you never had to wait very long to talk to someone.
  3. On-the-Spot Research: The event had an app where you could find links to the companies and their open positions.  After I talked to the top companies on my list, I explored other companies and got to talk to some really interesting people!
  4. Make a Schedule: I did not get to attend any of the workshops I wanted to because I was distracted by the event itself!  If you really want to attend a workshop, make sure you remind yourself about when it starts and ends.
  5. Dress Code: The event said to wear what you felt comfortable in…most people were in jeans.  It was a very laid-back atmosphere.  In fact, probably the most relaxed job fair I’ve ever been to.
  6. Beverages: If you want to drink alcohol, there was an open bar, but you had to bring your ID to get a wrist band.  In fact, I saw both candidates and employers drinking, which I have never seen at an event like this.  There was also free coffee, which was delicious, and also more my speed at a job fair.

Conclusion

Whether you are looking for a new position, or just to increase your network in the NYC area, I would suggest checking out their event next year.  I think it was well worth the price of the ticket!

Callback Hell, Express, and Koa

I would like to start off by saying that I love Node because it was enables me to write full-stack web applications built entirely in JavaScript.  A lot of applications only use JavaScript for the front-end of the app because that is what it was originally intended for, but with Node we can build entire applications!  Yay!  *raucous applause*

However, using Node on its own can be tricky.  When I was first learning how to set up my server-side using Node, my friend gave me this advice: “First lesson in using Node – use Express. ”  I was curious about what he meant, but forged forward, only to soon find myself in callback hell with a series of other issues.

Callback Hell

Callback functions are functions that are executed at a later point in time.  Because JavaScript allows for asynchronous behavior, Node uses callbacks a lot to help deal with this. Instead of having functions just return a value, they will wrap that result in a callback function.  Since functions can’t execute until all the parameters are evaluated, the callback will only fire once the asynchronous job it needs to do is complete.  Once you get used to callbacks they aren’t so bad, but your code can get complicated and hard to read if you don’t take steps to keep your code modular and clean.  Callback hell happens when you end up having callbacks within callbacks within callbacks.

Additionally, when using Node on its own, you have to worry about parsing POST requests in a certain way.  Frameworks have many different modules you can choose from to simplify this process.

Solution: Frameworks

Express

Express is a popular framework to use with Node.  Honestly, once I started using Express I have never gone back to creating an app using just Node by itself – there are just too many benefits.  Express makes routing, serving up your static files, and a host of other things super simple.  For instance, you can require body-parser, a middleware that helps you take care of parsing data easily.

Koa

Koa differs from Express because it uses generator functions, a feature in ES6 that I happen to like a lot!  Interestingly, the people who are behind Koa are the same people that made Express, and they set out to make a tool that would make creating the server side your app even easier.  Anything you can do with Express you can do with Koa, but getting used to generator functions their syntax can take a little time.

Conclusion

I’m realizing as I’m writing this that this blog post should/could actually be turned into about three or four posts specifically introducing Express and Koa specifics.

TL;DR: Simplify your life by using Express or Koa when using NodeJS.

 

Event Loop in JavaScript

Last week, I wrote about blocking vs. non-blocking code and the asynchronous nature of JavaScript.  This week I wanted to deep dive into the event loop to explain one way JavaScript is able to deal with asynchronous functions and enables it to have non-blocking behavior.

Runtime Environment

Many people use JavaScript for the client-side of their app, but it can also be used on the server-side while using NodeJS.  No matter where the code is being executed, i.e. its runtime environment, the event loop impacts the order that functions are called.

The JS engine goes through every line of code in your application and has to make decisions about when that code should be run.

The Infamous Call Stack

1928327_520276868890_2899_n

A stack!  Okay, so not a call stack, but delicious nonetheless.

A stack is a common data structure.  It functions exactly the stack of pancakes my dad makes for the whole family during the holidays – one at a time, people are able to remove a pancake from the top of the stack and to their plate.  A stack works the same way – data on top of the stack can be removed, as well as added to the top of the stack until it needs to be processed.

The call stack is a special kind of stack in the JS engine.  It processes one line of code at a time, and it executes whatever is on top of the stack.  As soon as that function is finished executing, it is removed from the stack, and the line that was previously second in line is pushed to the top and executed.

Callbacks and Event Queues

Say we arrive at a line of code that needs this asynchronous behavior.  Maybe you wrote a setTimeOut in your code to invoke a function at a later time, or perhaps a function that needs to get data from the database.  These are called callback functions and the JS engine has a special way of handling them.

Instead of putting them directly into the call stack, the JS engine puts them into an event queue where it waits to be executed.  Once the call stack is empty, it checks out the event queue and moves any function that is next in line to the call stack where it is executed.

Event Loop

The event loop is this process the JavaScript engine uses to execute functions on the call stack while also checking and eventually executing the next function in line in the event queue.  As the user interacts with an application, the call stack and event queues are constantly moving and cycling through the code and reacting in response to user behavior.  This keeps your program running smoothly!

While this is a very simplified version of what happens inside of a JavaScript engine, it is important information to keep in mind when writing code.

 

Blocking and Non-blocking Code

A Tale of Two Burger Joints

Restaurant #1

You wait in line so the cashier can take your order. There’s about 4 guests ahead of you, and an ever increasing line of people behind you. Once you place your order, they give you a pager that lights up and vibrates when your food is ready. You step off the line and wait with the other people who placed their order before you. Eventually your buzzer goes off, you take your tray of deliciousness from the counter, and squeeze past the lines of people waiting to find a table. This whole process takes about 20-30 minutes.

Restaurant #2

There are about 3 people in ahead of you in line. The cashier takes the order of the person in front. Then, after about 15 minutes, the customer’s food is ready and they take it and leave. Now there are only 2 people in front of you in line, and the same process happens: no other orders can be taken until the transaction is complete. You have to wait until everyone in front of you places their order, pays, and gets their food. The whole process takes a really long time.

So What?

You may be thinking that Restaurant #1 sounds like any fast food place, and that you would never see a place like Restaurant #2 in business.

What does this have to do with code? Code can be blocking, like Restaurant #2. Code can also be non-blocking, like Restaurant #1. Javascript by nature is non-blocking and allows asynchronous functions to exist. From what I’ve gathered, it can take a little getting used to if you are used to writing blocking code that has to go in a certain sequence.

Some actions that need to take place in a program are asynchronous by nature.  For example, say you need to retrieve data from a database – it takes time for your program to go to the database, get the data, and return it to the client side of your app.  Meanwhile, the rest of the page still needs to load.  It would not make for a good user experience for them to look at a blank screen while the database was accessed.  Usually, you might see most of the page loaded, and maybe a spinner or message that lets the user know they are waiting for the data.  When writing code in Javascript, it’s important to keep asynchronous operations in mind so that your application functions the way you want it to.