mcottondesign

Loving Open-Souce One Anonymous Function at a Time.

Why are coders afraid of writing code?

At a previous startup, I had a great idea that we could improve our product with realtime updates from the server. At the time we had implemented long-polling in the client. After failing to get everyone on board I decided to take a weekend to create an example with socket.io and Node.js.

On Monday morning I was proud to show it off around the company. The CEO liked the idea and agreed with me. Yay! We got on a call with the CTO and I proudly explained my work.

He looked at it, then asked me to go into further detail about my choices. How did I implement this? Tell him about websockets? How can we roll it into productions? What does it look like at the HTTP layer? Well, I could sort of answer his questions. I had to confess that I really didn't know and I used socket.io and it uses websockets.

He probed a little more and then started to get impatient. I couldn't believe that he had become dismissive. And worse, the CEO followed his lead. I hadn't figured it out yet but I had made a critical mistake.

Would we now be implementing the socket.io abstraction? Is this what we want? What trade off does that mean for us? Is this the only way?

I was afraid to learn how the technology actually worked and I was asking the company to take in several major dependencies in order to implement this.

I stubbornly maintained my position. Why would we skip over an existing answer? I had proved it was working. They could see it working. Why were they not as excited as I am?

We were at an impasse and I still hadn't figured it out yet.

Fast-forward to now, and I am a first time CTO. I've spent a lot of time thinking about lessons learned from my friend, the CTO in the previous story. I do my best to consider why he made the choices he did.

I'm interviewing candidates and I'm starting to see a similar pattern with other developers. They gladly accept dependencies and baggage that limits options and forces a direction. They'll make a terrible choice to avoid doing something that makes then uncomfortable.

Coders are afraid of writing code.

A fiend of mine recently described how their company need to do multiple events from an external webhook. Not enough people understand a simple web server so it got pushed to DevOps to build using salt. Salt isn't a horrible answer but when he was telling me about this I could see a little of my inexperienced self in the story.

On another project I was dealing with an Indian iOS developer that needed to sort an array. They fought me when I filed a bug that the sort order wasn't correct. A week later they had it working but had implement it in the worst way possible. They had pulled in SQLite as a new dependency and were using that to sort the array. When I saw this I was furious. They were very cheap agency but after this they weren't worth keeping around at any price.

In conclusion, we ended up implementing websockets. I was the champion for this feature and after our CTO guided me in the right direction. We read through the spec for websockets, we came up with the right fallback strategy, and the product improved significantly. We didn't have to blindly add dependencies, we understood the technology we wanted touse, and we found the right fit for our needs.

What is the weirdest bug you've ever dealt with?

I was asked this once at an informal interview and didn't have an immediate answer. I told a story about the time I was trying to debug RF signals and the ultimate cause was that there was another remote using the same hardware ID. It was extremely unprobabel, but it turned out to be the cause.

Since then I've had more time to reflect on how I would answer that question if I was asked again.

The weirdest bugs I've ever dealt with all have a common theme, distributed systems. In my last two startups we have used distributed architectures. This has caused somewhat simple tasks to become very difficult.

In the old days of a monolithic app server (Django, Rails, PHP) all the logs were in one place. As you scaled you would need to add a load balancer and then spread the requests over multiple app servers. Reasoning about the path an HTTP request took was still fairly simple. At this level a relational database is perfectly able to handle the load.

In a distributed system, HTTP is used not just for the inital request but also for communication between servers. This means you now have to deal with internal HTTP responses and they have different meansing from the external request/response that you're used to. For example, what to do when an internal call to the cache returns a 401, 403, or 502? Do the unauthorized/forbidden calls mean that the external's request is 401/403 or does this mean your app server is? What about 502 Gateway Timeout? You now have to deal with timing and latency between internal services.

Distributed computer and networking are a reality of life for most new projects. It is something we'll have to learn and deal with. In the meantime, it is also the source of my weirdest bugs.

HUVR Truck

Showing off our truck on a nice, sunny afternoon. Shot with a Phantom 3 Pro and an Inspire 1.

Leaving Eagle Eye and moving to HUVRdata

I am excited to announce that I have accepted a new position with HUVRDATA. After an eight week transition I am thrilled to get started. I wI'll be their CTO and will guide them into becoming the leader in drone based inspections.

It was surprisingly hard to leave Eagle Eye and specifically, let the team know what was next for me. I lost sleep over this decision but we were able to make it through. I loved working with them and it was filled with great memories.

I've been working with Bob and Ben at HUVR since the start of the year. Thinks really kicked into gear when they closed a funding round in August. Right now it is time to mash the accelerator and I would have regretted letting them go on without me. This is an area I am very passionate about, have the skills to do it, and am fully on-board to make this successful.

Small Improvements Over Time

Improving takes time and patience and lots of near-misses. There is usually a narrow window in to capture the ideal scene and if you aren't in position and ready you will miss it. I continue to be 10 minutes late, out of battery or space on the SD card when I realize this is the shot I wanted.

In this case, I wanted to try the waters before going again the next day. The challenges of shooting downtime is that you attract too much attention to play around with expensive equipment as a crowd gathers.

Austin after Sunset

Pushing the distance out a little further and keeping it low across the water. Experimenting with Final Cut Pro X to get the titles and transition.

Out on the a Bat Cruise

I had a terrific time with @LSRiverboats getting this footage. This was captured in 4k@30fps but rendered to 720p for Vimeo. The initial shot was digitally zoomed in and the reset is right off the camera. Some minor exposure correction was done in post to make every match.

This was a lot of fun but extremely challenging to operate from a boat. There were two boats, fading daylight, bats and other groups to contend with. Thankfully everything worked out and I am happy with the results.

Getting Better

I've been trying for quite some time to get this shot. It was interesting because this was also my longest distance flight. I only went 1000ft' up the river away and it took a lot of squinting to keep it in sight.

Another Try at an Old Location

This was taken out front of the Austin City Power Plant and is using the new manual camera controls of the Phantom 3 Pro.

It is hard to go back to previews locations and try to get a fresh take. The challenge is to that you are no longer looking at it with fresh eyes. This is made worse if you liked the results from previous tries.

I am challenging myself to throw away my previous work. All locations (except the Capital) are fair game. I am also going to not get hung up on having done the shot before. I will also shamelessly steal good ideas for others. I am still in the beginner phase and am in no position to be selective.

Smooth Sailin'

This is video from my first flight with the new Phantom 3 Pro. I picked it up earlier today and ran it through its paces. So far, it is a significant improvement from the Phantom 2 Vision +. The iOS app is the major differentiator. All the other features are in support of the app. The remote is the same as the Inspire 1 and feels much more refined. Common features can be done using hard buttons and there is less time spent pawing at the app. I am a big fan of physical buttons and not having to look at what I'm pushing.

I have (foolishly) upgraded my iPad Retina mini to beta versions of iOS 9. The DJI app isn't playing nicely with the new HD video streaming. I can't hold this against them.

Setbacks and Crashes

This is an older video that I hadn't posted yet. It is from a couple months ago and has nothing to do with the rest of the post.

I had a crash earlier in the week that has destroyed my Phantom 2. Almost immediately after take it it lost all power and unceremoniously fell from the sky. My first reaction was shock. I don't know if the battery wasn't seated all the way or it experienced a different malfunction.

After some time to think, my shock turned to gratitude. Gratitude that this wasn't over water, wasn't for a client, wasn't over a road, wasn't over a crowd. I don't fly over crowds but it is always tempting to push the boundaries.

Pushing the boundaries is what makes this emerging field exciting. Getting these images end of being expensive when you look at the output that gave from the money invested.

I can't wait to announce my next aircraft.

Getting more cinematic

The goal was to make a short compilation from several different shots. I wanted something visually interesting without spending hours on each shot. This was all done on just one battery. Except for the last shot, we only took one pass at each shot and didn't review anything. I had several more shots in mind, but will save those for another compilation. I enjoying improvising and the preasure of just starting the camera.

Sunrise Panorama

This is a composite image from several individual shots. It was taken a peaceful Sunday morning just after sunrise. I really enjoy Sunday mornings photo shoots. Austin is still recovering from its collective hangover, the roads are empty, and the sun isn't scorching hot, yet.

There is something more challenging about get a sunrise picture. Sunsets are easy, I can tell how the sunset will turn out on my drive home. There is a very little risk that the weather will change between packing up setting up. On the downside, flying at sunset garners more attention and then you are working home in the dark (with expensive equipment).

Most importantly, after a nice Sunday morning out flying you can always get breakfast tacos on the way back to bed. Let me know if you are interested in this image.

Making Progress

I'm starting to make progress with my ariel photography. The images and video are making me cringe less. I'm starting very basic and working up slowly. Instead of just cranking up fancy effects in LightRoom or adding some razzle-dazzle from Final Cut Pro. Right now it is about making something that works well as Desktop wallpaper.

This blog is the new starting point for a gallery. The goal is more about progress instead of a store of available prints. (That's not to stay I won't sell out and turn this into a store). If there is something that catches your fancy, fell free to reach out.

Putting in the hours

One of the great advantages to sharing your beginner work is that you are able to see rapid progress. The initial learning curve is usually pretty quick. Making a Vimeo and YouTube channel and posting to reddit provide quick feedback. A lot of the feedback will be critical of what you are doing, but this is all part of learning.

Once you reach a certain level of achievement you reach a plateau and growth levels off. This is usually a key area of attrition. The funnel tightens and many people are happy to stop getting better. It is important to identify what the next achievement will look like and keep moving towards it. For me, it was being included in an actual video production.

If you are interested, get in touch with Brent at In Focus Video

Picking a new hobby and starting over

I'm trying out a new hobby, that has been a casual interest of mine. This year is going to be the year of GoPro photography. It is a kind of weird idea, but the idea is really simple. I've bought the Hero 4 Black so there isn't a better model I can blame for my shortcomings. There is also a wide enough group of users posting their content that I have a significant yard stick to compare myself to.

What have I learned so far?

I've learned that it is easy to have an opinion about what looks good but it is very hard to produce something that I am proud to share. This is the challenge of creative work. My initial work is an embarrassment so far. It is well below the standard I want to produce but this is exactly the point.

What do I want to get out of this?

I want to produce work that I am proud to share. Also, I want to be more comfortable sharing my work. Expanding the things I am comfortable doing without much notice is a sign that I'm progressing towards being an interesting human being.

The same things that hold me back from dancing at weddings are the same things that hold me back with my creative work. This way I'll have something interesting to show people who are also not dancing at weddings.

Translating text in Backbone.js templates

We recently added Japanese support to our webapp. When implementing it, we decided on the most straight-forward method of converting it at run-time by adding a helper method to the view.

The first step was to extend Backbone.View to include a translate method (abbreviated as 't' to reduce noise in the template). It performs a simple dictionary lookup. It falls back to English if a suitable translation can not be found.

BaseView = Backbone.View.extend({
     t: function(str, skip) {
        // check if the browser/system language is Japanese
        var lang = 'ja'
        var dictionary = {
                'US': {},
                'ja': {
                    'Dashboard': 'ダッシュボード',
                    'Layouts': 'レイアウト',
                    'Cameras': 'カメラ',
                    'Users': 'ユーザー',
                    'Map': 'マップ',
                    'Installer Tools': 'インストールツール',
                    'Status': '状態'
                }
        }

        return dictionary[lang][str]== undefined ? str : dictionary[lang][str];
    }
});

And the specific view needs to be changed.

AddCameraMessageView = Backbone.View.extend({

gets changed to be

AddCameraMessageView = BaseView.extend({

The individual templates now can use the new translate helper method on any string.

<tr>
    <td>Device</td>
    <td>Name</td>
    <td>Status</td>
</tr>

gets changed to be

<tr>
    <td><%= this.t('Device') %></td>
    <td><%= this.t('Name') %></td>
    <td><%= this.t('Status') %></td>
</tr>

The next iteration will perform this translation at the build phase by writing a grunt plugin instead of at run-time. This will remove any rendering time penalty for our users.

Testing out a new project

I'm working on a new project that allows people to self-host a "proxy" server on Google App Engine that pulls from the Eagle Eye Networks API (authenticated) and then makes the preview images available embedded into their public website (unauthenticated). In the case below, I just dropped in an iframe from the app engine instance.

Talk at Austin Python Meetup 8/14

I had a lot of fun speaking at the Austin Python Meetup. My presentation was on the current options available to those who want use python with embedded hardware. It was a great group of people and I got the chance to bring some toys.

Presentation is here

Using animated GIFs to answer support questions

Videos are great at demonstrating a feature, but they are a slow and clunky experience. Animated GIFs are quick and lightweight, and no voice narration. I used Licecap to capture these videos. Embedding them is as easy as an img tag

How do I add a bridge?

How do I add a camera?

How do I get to camera settings?

How do I change camera resolution?

How do I view cameras?

How do I create a layout?

How do I resize a layout?

How I use Frontend build tools and Grunt.js

Build tools are traditionally the realm of backend systems. Recently, as frontend systems have gotten more complex, JavaScript specific tools have emerged. There are plenty of options and it would be impossible to keep up with them all. I wanted to explain how we make use of Grunt.js and what it does for us.

What are Frontend build tools

There are several tasks that need to be done before releasing a modern webapp. The JavaScript needs to be concatenated into a single file to improve page speed. This should also be done with the CSS and all assets should be gzipped. Files should be minified and uglified as appropiate and in the case of html, prettyprinted.

Concatenating Files

Web browsers download all the included script files before parsing and evaluating them. It is best to have the least number of includes and the smallest includes possible.

On the other hand, in order to develop between team members and keep organized, it is best to break separate code into separate files. Combining them all when getting ready for a release could be a tedious and error-prone process. This is something that Grunt.js excels at. Using the standard 'concat' task you can specify the files and order that you'd like them mashed together. Doing this from the start lets you use the samething in development that you'll release in production.

We breakout our JavaScript into separate folders and then have an individual file for each 'class'. All the Views get rolled together along with Models, Collections, Templates and dependencies.

JS Folder
   |
   |- Views
   |    |
   |    |- AccountSummaryView.js
   |    |- AccountView.js
   |    |- AddAccountView.js
   |    |- AddNewCameraView.js
   |    |- ...
   |
   |- Models
   |    |
   |    |- Account.js
   |    |- Bridge.js
   |    |- Camera.js
   |    |- ....
   |
   |- ...

This all becomes

JS Folder
   |
   |- backbone.views.js
   |- backbone.models.js
   |- ...

and then those become

JS Folder
   |
   |- backbone.combined.js

This strategy also makes it easy to stuff templates into the index.html page so they are already in the DOM and speed up selector time.

Cache Busting

One of the problems with using the same file names is that the browser is cache the files. Some browser are more aggressive about caching then others. This is great for files that don't change but for your new concatenated file it is very frustrating. Thanksfully Grunt.js has a hashres module that will look at your index.html and push a file hash into the file name.

<script src="_js/backbone.combined.js"></script>

turns into

<script src="_js/backbone.combined_58cde93c.js"></script>

This is terrific because as files are combined and generated only truely unique files will bust the cache. Nginx is smart enough to filter out the hash in the filename and serve the correct file but the browser will think it new and different.

Watching for changes

All of this functionality would be ignored if you had to tab over to the console and re-run the grunt command over and over. Included with Grunt.js is the watch module that lets you define the file patterns that it should watch. When the files change it will automagically run the grunt tasks and you can safely forget about it.

Conclusion

There is a lot more to learn about Grunt.js, hopefully this has got you excited. Go dig in and try it out.

First Demo of my RFID integration

In order to show off the EEN API I made an example access control system out of a Raspberry PI, RFID reader and Node.js

Node.js was a natural fit for this project because of how easy it is to handle events from the RFID reader, add realtime support through websockets and pull data out of Eagle Eye Networks with the een module.

All the important changes can be made from config.js in the root directory. From there you can define badges and doors. Each door has its own reader which is defined by the serial port it is connected to. The doors define the rules of which badges are authorized to open. Adding time checks would be trivial to extend.

On startup it opens the serial ports and begins listening for strings that match the know badge format. Next, it starts a webserver and waits for a websocket connection from a client. Once the client connects it sends the current state to allow the client to get in sync.

New events are realtime, the template is rendered on the client to keep server traffic down. The timestamp is recorded and a preview image is fetched. The internal webserver acts as a authentication proxy for images and videos. The image and video link could then be saved for further processing.

You can find the project on github here: node-rfid

JavaScript Journal - Getting Started with Infinite Scrolling

Infinite scrolling is a really cool effect and a great use of AJAX. It also has some weird side-effects that you'll need to be aware of. This is the starting point I used in the event version of the Gallery Viewer. The major parts are throttling the scroll event and detecting the scrolling direction. Because I am loading images I want to get started on the request as soon as possible, or I would wait until they were close to the bottom of the page before loading. There is a also a little <div> with a message that keeps jumping to the bottom of the page.

    // keep scroll position so we can use it 
    // later to detect scroll direction
    var oldScrollPosition = 0;

    // throttle down the scroll events to once ever 1.5 seconds
    var throttled = _.throttle(function(e) {
        var newScrollPosition = $(this).scrollTop();
        // check if they scrolled down the page
        if(newScrollPosition > oldScrollPosition) {
            self.pullVideoList(self.num);
            oldScrollPosition = newScrollPosition;
        }
    }, 1500);
    $(window).scroll(throttled);

Working with the API: getting previews

Introduction

This blog post is an example of how you can use the Eagle Eye Networks API to embed the preview stream where ever you want.

Background

The API makes it very easy to get the preview stream for all the cameras in an account. The preview stream is a series of JPEG images and requires authentication. Our code today will show how to use the API to provide the images on a webpage without authentication.

Step 1:

We are going to use the code at mcotton/watcher to run a Node.js server that will login, subscribe to the poll stream, notify the client and proxy the image requests. We could remove the poll stream to make the example very basic but it is worth going through it now.

Download or clone the git repository.

Step 2:

You will need to have Node.js installed (current version is v0.10.26). Once it is installed, open up the command line in the same directory as the example code, you can install the dependencies by running

npm install

Edit the file named config.js, replace 'your_username' and 'your_password' with your username and password

module.exports = {
    username   =       'your_username',
    password   =       'your_password'
}

Save the file and start the server

npm start

Step 3:

You can now open a browser at go to localhost:3000 and you will see previews show up as the become available on the server.

Because we are subscribed to the poll stream we are only fetching previews when they are available. This makes it very efficient with the bandwidth.

Conclusion:

You can now treat the image URLs just like static assets on your server. You can put it behind your own firewall or you can make it publicly available. In both cases your username and password are safely stored on the server and never sent to the client.

Getting updated properties

Sometimes you want to get an updated property . In this case, I am animating an image to make it 10% bigger when I hover over it. I also have a slider that allows the grid sizing to change.

$(img.currentTarget).animate({
    'position': 'absolute',
    'height': (function() { return $('#slider').slider('value') * 1.1 + '%' })(),
    'width': (function() { return $('#slider').slider('value') * 1.1 + '%' })(),
    'z-index': '10',
    'margin': '-15px -15px -15px -15px'},
    0,
    function() {}
})

Here I am replacing a static property with a function that can return new information. It is still accessed in the same way but behaves the way I wanted.