mcottondesign

Loving Open-Souce One Anonymous Function at a Time.

What is the weirdest bug you've ever dealt with?

I was asked this once at an informal interview and didn't have an immediate answer. I told a story about the time I was trying to debug RF signals and the ultimate cause was that there was another remote using the same hardware ID. It was extremely unprobabel, but it turned out to be the cause.

Since then I've had more time to reflect on how I would answer that question if I was asked again.

The weirdest bugs I've ever dealt with all have a common theme, distributed systems. In my last two startups we have used distributed architectures. This has caused somewhat simple tasks to become very difficult.

In the old days of a monolithic app server (Django, Rails, PHP) all the logs were in one place. As you scaled you would need to add a load balancer and then spread the requests over multiple app servers. Reasoning about the path an HTTP request took was still fairly simple. At this level a relational database is perfectly able to handle the load.

In a distributed system, HTTP is used not just for the inital request but also for communication between servers. This means you now have to deal with internal HTTP responses and they have different meansing from the external request/response that you're used to. For example, what to do when an internal call to the cache returns a 401, 403, or 502? Do the unauthorized/forbidden calls mean that the external's request is 401/403 or does this mean your app server is? What about 502 Gateway Timeout? You now have to deal with timing and latency between internal services.

Distributed computer and networking are a reality of life for most new projects. It is something we'll have to learn and deal with. In the meantime, it is also the source of my weirdest bugs.

HUVR Truck

Showing off our truck on a nice, sunny afternoon. Shot with a Phantom 3 Pro and an Inspire 1.

Leaving Eagle Eye and moving to HUVRdata

I am excited to announce that I have accepted a new position with HUVRDATA. After an eight week transition I am thrilled to get started. I wI'll be their CTO and will guide them into becoming the leader in drone based inspections.

It was surprisingly hard to leave Eagle Eye and specifically, let the team know what was next for me. I lost sleep over this decision but we were able to make it through. I loved working with them and it was filled with great memories.

I've been working with Bob and Ben at HUVR since the start of the year. Thinks really kicked into gear when they closed a funding round in August. Right now it is time to mash the accelerator and I would have regretted letting them go on without me. This is an area I am very passionate about, have the skills to do it, and am fully on-board to make this successful.

Small Improvements Over Time

Improving takes time and patience and lots of near-misses. There is usually a narrow window in to capture the ideal scene and if you aren't in position and ready you will miss it. I continue to be 10 minutes late, out of battery or space on the SD card when I realize this is the shot I wanted.

In this case, I wanted to try the waters before going again the next day. The challenges of shooting downtime is that you attract too much attention to play around with expensive equipment as a crowd gathers.

Austin after Sunset

Pushing the distance out a little further and keeping it low across the water. Experimenting with Final Cut Pro X to get the titles and transition.

Out on the a Bat Cruise

I had a terrific time with @LSRiverboats getting this footage. This was captured in 4k@30fps but rendered to 720p for Vimeo. The initial shot was digitally zoomed in and the reset is right off the camera. Some minor exposure correction was done in post to make every match.

This was a lot of fun but extremely challenging to operate from a boat. There were two boats, fading daylight, bats and other groups to contend with. Thankfully everything worked out and I am happy with the results.

Getting Better

I've been trying for quite some time to get this shot. It was interesting because this was also my longest distance flight. I only went 1000ft' up the river away and it took a lot of squinting to keep it in sight.

Another Try at an Old Location

This was taken out front of the Austin City Power Plant and is using the new manual camera controls of the Phantom 3 Pro.

It is hard to go back to previews locations and try to get a fresh take. The challenge is to that you are no longer looking at it with fresh eyes. This is made worse if you liked the results from previous tries.

I am challenging myself to throw away my previous work. All locations (except the Capital) are fair game. I am also going to not get hung up on having done the shot before. I will also shamelessly steal good ideas for others. I am still in the beginner phase and am in no position to be selective.

Smooth Sailin'

This is video from my first flight with the new Phantom 3 Pro. I picked it up earlier today and ran it through its paces. So far, it is a significant improvement from the Phantom 2 Vision +. The iOS app is the major differentiator. All the other features are in support of the app. The remote is the same as the Inspire 1 and feels much more refined. Common features can be done using hard buttons and there is less time spent pawing at the app. I am a big fan of physical buttons and not having to look at what I'm pushing.

I have (foolishly) upgraded my iPad Retina mini to beta versions of iOS 9. The DJI app isn't playing nicely with the new HD video streaming. I can't hold this against them.

Setbacks and Crashes

This is an older video that I hadn't posted yet. It is from a couple months ago and has nothing to do with the rest of the post.

I had a crash earlier in the week that has destroyed my Phantom 2. Almost immediately after take it it lost all power and unceremoniously fell from the sky. My first reaction was shock. I don't know if the battery wasn't seated all the way or it experienced a different malfunction.

After some time to think, my shock turned to gratitude. Gratitude that this wasn't over water, wasn't for a client, wasn't over a road, wasn't over a crowd. I don't fly over crowds but it is always tempting to push the boundaries.

Pushing the boundaries is what makes this emerging field exciting. Getting these images end of being expensive when you look at the output that gave from the money invested.

I can't wait to announce my next aircraft.

Getting more cinematic

The goal was to make a short compilation from several different shots. I wanted something visually interesting without spending hours on each shot. This was all done on just one battery. Except for the last shot, we only took one pass at each shot and didn't review anything. I had several more shots in mind, but will save those for another compilation. I enjoying improvising and the preasure of just starting the camera.

Sunrise Panorama

This is a composite image from several individual shots. It was taken a peaceful Sunday morning just after sunrise. I really enjoy Sunday mornings photo shoots. Austin is still recovering from its collective hangover, the roads are empty, and the sun isn't scorching hot, yet.

There is something more challenging about get a sunrise picture. Sunsets are easy, I can tell how the sunset will turn out on my drive home. There is a very little risk that the weather will change between packing up setting up. On the downside, flying at sunset garners more attention and then you are working home in the dark (with expensive equipment).

Most importantly, after a nice Sunday morning out flying you can always get breakfast tacos on the way back to bed. Let me know if you are interested in this image.

Making Progress

I'm starting to make progress with my ariel photography. The images and video are making me cringe less. I'm starting very basic and working up slowly. Instead of just cranking up fancy effects in LightRoom or adding some razzle-dazzle from Final Cut Pro. Right now it is about making something that works well as Desktop wallpaper.

This blog is the new starting point for a gallery. The goal is more about progress instead of a store of available prints. (That's not to stay I won't sell out and turn this into a store). If there is something that catches your fancy, fell free to reach out.

Putting in the hours

One of the great advantages to sharing your beginner work is that you are able to see rapid progress. The initial learning curve is usually pretty quick. Making a Vimeo and YouTube channel and posting to reddit provide quick feedback. A lot of the feedback will be critical of what you are doing, but this is all part of learning.

Once you reach a certain level of achievement you reach a plateau and growth levels off. This is usually a key area of attrition. The funnel tightens and many people are happy to stop getting better. It is important to identify what the next achievement will look like and keep moving towards it. For me, it was being included in an actual video production.

If you are interested, get in touch with Brent at In Focus Video

Picking a new hobby and starting over

I'm trying out a new hobby, that has been a casual interest of mine. This year is going to be the year of GoPro photography. It is a kind of weird idea, but the idea is really simple. I've bought the Hero 4 Black so there isn't a better model I can blame for my shortcomings. There is also a wide enough group of users posting their content that I have a significant yard stick to compare myself to.

What have I learned so far?

I've learned that it is easy to have an opinion about what looks good but it is very hard to produce something that I am proud to share. This is the challenge of creative work. My initial work is an embarrassment so far. It is well below the standard I want to produce but this is exactly the point.

What do I want to get out of this?

I want to produce work that I am proud to share. Also, I want to be more comfortable sharing my work. Expanding the things I am comfortable doing without much notice is a sign that I'm progressing towards being an interesting human being.

The same things that hold me back from dancing at weddings are the same things that hold me back with my creative work. This way I'll have something interesting to show people who are also not dancing at weddings.

Translating text in Backbone.js templates

We recently added Japanese support to our webapp. When implementing it, we decided on the most straight-forward method of converting it at run-time by adding a helper method to the view.

The first step was to extend Backbone.View to include a translate method (abbreviated as 't' to reduce noise in the template). It performs a simple dictionary lookup. It falls back to English if a suitable translation can not be found.

BaseView = Backbone.View.extend({
     t: function(str, skip) {
        // check if the browser/system language is Japanese
        var lang = 'ja'
        var dictionary = {
                'US': {},
                'ja': {
                    'Dashboard': 'ダッシュボード',
                    'Layouts': 'レイアウト',
                    'Cameras': 'カメラ',
                    'Users': 'ユーザー',
                    'Map': 'マップ',
                    'Installer Tools': 'インストールツール',
                    'Status': '状態'
                }
        }

        return dictionary[lang][str]== undefined ? str : dictionary[lang][str];
    }
});

And the specific view needs to be changed.

AddCameraMessageView = Backbone.View.extend({

gets changed to be

AddCameraMessageView = BaseView.extend({

The individual templates now can use the new translate helper method on any string.

<tr>
    <td>Device</td>
    <td>Name</td>
    <td>Status</td>
</tr>

gets changed to be

<tr>
    <td><%= this.t('Device') %></td>
    <td><%= this.t('Name') %></td>
    <td><%= this.t('Status') %></td>
</tr>

The next iteration will perform this translation at the build phase by writing a grunt plugin instead of at run-time. This will remove any rendering time penalty for our users.

Testing out a new project

I'm working on a new project that allows people to self-host a "proxy" server on Google App Engine that pulls from the Eagle Eye Networks API (authenticated) and then makes the preview images available embedded into their public website (unauthenticated). In the case below, I just dropped in an iframe from the app engine instance.

Talk at Austin Python Meetup 8/14

I had a lot of fun speaking at the Austin Python Meetup. My presentation was on the current options available to those who want use python with embedded hardware. It was a great group of people and I got the chance to bring some toys.

Presentation is here

Using animated GIFs to answer support questions

Videos are great at demonstrating a feature, but they are a slow and clunky experience. Animated GIFs are quick and lightweight, and no voice narration. I used Licecap to capture these videos. Embedding them is as easy as an img tag

How do I add a bridge?

How do I add a camera?

How do I get to camera settings?

How do I change camera resolution?

How do I view cameras?

How do I create a layout?

How do I resize a layout?

How I use Frontend build tools and Grunt.js

Build tools are traditionally the realm of backend systems. Recently, as frontend systems have gotten more complex, JavaScript specific tools have emerged. There are plenty of options and it would be impossible to keep up with them all. I wanted to explain how we make use of Grunt.js and what it does for us.

What are Frontend build tools

There are several tasks that need to be done before releasing a modern webapp. The JavaScript needs to be concatenated into a single file to improve page speed. This should also be done with the CSS and all assets should be gzipped. Files should be minified and uglified as appropiate and in the case of html, prettyprinted.

Concatenating Files

Web browsers download all the included script files before parsing and evaluating them. It is best to have the least number of includes and the smallest includes possible.

On the other hand, in order to develop between team members and keep organized, it is best to break separate code into separate files. Combining them all when getting ready for a release could be a tedious and error-prone process. This is something that Grunt.js excels at. Using the standard 'concat' task you can specify the files and order that you'd like them mashed together. Doing this from the start lets you use the samething in development that you'll release in production.

We breakout our JavaScript into separate folders and then have an individual file for each 'class'. All the Views get rolled together along with Models, Collections, Templates and dependencies.

JS Folder
   |
   |- Views
   |    |
   |    |- AccountSummaryView.js
   |    |- AccountView.js
   |    |- AddAccountView.js
   |    |- AddNewCameraView.js
   |    |- ...
   |
   |- Models
   |    |
   |    |- Account.js
   |    |- Bridge.js
   |    |- Camera.js
   |    |- ....
   |
   |- ...

This all becomes

JS Folder
   |
   |- backbone.views.js
   |- backbone.models.js
   |- ...

and then those become

JS Folder
   |
   |- backbone.combined.js

This strategy also makes it easy to stuff templates into the index.html page so they are already in the DOM and speed up selector time.

Cache Busting

One of the problems with using the same file names is that the browser is cache the files. Some browser are more aggressive about caching then others. This is great for files that don't change but for your new concatenated file it is very frustrating. Thanksfully Grunt.js has a hashres module that will look at your index.html and push a file hash into the file name.

<script src="_js/backbone.combined.js"></script>

turns into

<script src="_js/backbone.combined_58cde93c.js"></script>

This is terrific because as files are combined and generated only truely unique files will bust the cache. Nginx is smart enough to filter out the hash in the filename and serve the correct file but the browser will think it new and different.

Watching for changes

All of this functionality would be ignored if you had to tab over to the console and re-run the grunt command over and over. Included with Grunt.js is the watch module that lets you define the file patterns that it should watch. When the files change it will automagically run the grunt tasks and you can safely forget about it.

Conclusion

There is a lot more to learn about Grunt.js, hopefully this has got you excited. Go dig in and try it out.

First Demo of my RFID integration

In order to show off the EEN API I made an example access control system out of a Raspberry PI, RFID reader and Node.js

Node.js was a natural fit for this project because of how easy it is to handle events from the RFID reader, add realtime support through websockets and pull data out of Eagle Eye Networks with the een module.

All the important changes can be made from config.js in the root directory. From there you can define badges and doors. Each door has its own reader which is defined by the serial port it is connected to. The doors define the rules of which badges are authorized to open. Adding time checks would be trivial to extend.

On startup it opens the serial ports and begins listening for strings that match the know badge format. Next, it starts a webserver and waits for a websocket connection from a client. Once the client connects it sends the current state to allow the client to get in sync.

New events are realtime, the template is rendered on the client to keep server traffic down. The timestamp is recorded and a preview image is fetched. The internal webserver acts as a authentication proxy for images and videos. The image and video link could then be saved for further processing.

You can find the project on github here: node-rfid

JavaScript Journal - Getting Started with Infinite Scrolling

Infinite scrolling is a really cool effect and a great use of AJAX. It also has some weird side-effects that you'll need to be aware of. This is the starting point I used in the event version of the Gallery Viewer. The major parts are throttling the scroll event and detecting the scrolling direction. Because I am loading images I want to get started on the request as soon as possible, or I would wait until they were close to the bottom of the page before loading. There is a also a little <div> with a message that keeps jumping to the bottom of the page.

    // keep scroll position so we can use it 
    // later to detect scroll direction
    var oldScrollPosition = 0;

    // throttle down the scroll events to once ever 1.5 seconds
    var throttled = _.throttle(function(e) {
        var newScrollPosition = $(this).scrollTop();
        // check if they scrolled down the page
        if(newScrollPosition > oldScrollPosition) {
            self.pullVideoList(self.num);
            oldScrollPosition = newScrollPosition;
        }
    }, 1500);
    $(window).scroll(throttled);

Working with the API: getting previews

Introduction

This blog post is an example of how you can use the Eagle Eye Networks API to embed the preview stream where ever you want.

Background

The API makes it very easy to get the preview stream for all the cameras in an account. The preview stream is a series of JPEG images and requires authentication. Our code today will show how to use the API to provide the images on a webpage without authentication.

Step 1:

We are going to use the code at mcotton/watcher to run a Node.js server that will login, subscribe to the poll stream, notify the client and proxy the image requests. We could remove the poll stream to make the example very basic but it is worth going through it now.

Download or clone the git repository.

Step 2:

You will need to have Node.js installed (current version is v0.10.26). Once it is installed, open up the command line in the same directory as the example code, you can install the dependencies by running

npm install

Edit the file named config.js, replace 'your_username' and 'your_password' with your username and password

module.exports = {
    username   =       'your_username',
    password   =       'your_password'
}

Save the file and start the server

npm start

Step 3:

You can now open a browser at go to localhost:3000 and you will see previews show up as the become available on the server.

Because we are subscribed to the poll stream we are only fetching previews when they are available. This makes it very efficient with the bandwidth.

Conclusion:

You can now treat the image URLs just like static assets on your server. You can put it behind your own firewall or you can make it publicly available. In both cases your username and password are safely stored on the server and never sent to the client.

Getting updated properties

Sometimes you want to get an updated property . In this case, I am animating an image to make it 10% bigger when I hover over it. I also have a slider that allows the grid sizing to change.

$(img.currentTarget).animate({
    'position': 'absolute',
    'height': (function() { return $('#slider').slider('value') * 1.1 + '%' })(),
    'width': (function() { return $('#slider').slider('value') * 1.1 + '%' })(),
    'z-index': '10',
    'margin': '-15px -15px -15px -15px'},
    0,
    function() {}
})

Here I am replacing a static property with a function that can return new information. It is still accessed in the same way but behaves the way I wanted.

Making a wireless security system

Our condo recently had some flood damaged and their are workers coming in and out to do repairs. My wife and I both work and do not feel comfortable with strangers in our house without us. We don't have the needed wiring in our hallway and my wife didn't want to see any wires run down the wall. So I made a wired camera wireless with parts I had laying around.

First I needed a wireless to wired bridge. There are several specialty parts available that do this but one of the cheapest and easiest to use is the Apple Airport Express. When you join it to a wireless networks, the ethernet port becomes a LAN port.
Airport Express

I also had an older Axis M1011 wired camera. Any IP camera will work through the wireless bridge, but make sure you have a way to power it.

Axis M1011

You can use the Axis clamp bracket and the power supply to power and mount the unit. Plug it all in and it will work just as if it were wired.
finished back view

finished front view

finished front view