Podcast: Play in new window | Download
Subscribe: iTunes | Android | RSS
Justin and Jason talk about Udi coming to America, whether Justin should use the $$ framework for Digedu, Justin’s theory on show titles, the role of talent in reaching the highest level’s of success, the last Catalyst session, the TechZing bulletin board, Jason and Guyon’s solution to the Uber hackathon, how Upverter is coming to the Pasadena Science & Technology Expo and how Jason is going to attend the upcoming Upverter / YC hardware hackathon, how Jason was quoted in the press, how Justin picked up high-paying consulting work, the logo for Jason’s secret project, why Jason thinks it’s time to learn Go if you’re a consultant, the Zerigo DNS service, how Justin is using NewsBlur’s RSS API, the Splittsville and MarkITx iPhone apps, why Tony Stark is the ultimate engineering role model, and an update on the AnyFu recruiting process.
I think, um.. iTunes really “liked the title” and downloaded that.
@Tolga… oh yea of little faith… I’m going for the people who don’t auto download and click the “get” button 😉 Anyway, just a test…
This has to be one of the funniest episodes. Nice work guys.
Okay now that you’re really going to have a LIVE show I need to know how much the tickets are going to be. It’s not fair that Udi gets to go and none of the rest of us.
Guys, halfway through the show, very entertaining 😉
@justin I download (via the website) all the discussion shows regardless of the title. I probably listen to 70% of interviews depending on my interest in the subject. You could easily use some split testing software to test your theories. Would be interested to hear the results.
@jason thanks for the shout out! It ill be interesting to see what comes of rubbing up against the TechZing luck surface area 🙂 I’m also excited about the chance to have a chrome book named after me as well :-p Please put me on a layaway plan, first instalment is on it’s way.
@justin you are right, the “TH” at the end of McGrath is silent, even down here in Australia.
Your discussion about your musical abilities reminded me of Amadeus, a play I studied years ago in high school. One of the things that always stuck with me from that play was that Salieri’s (the protagonist) main burden was that whilst all those around him thought his music was amazing, he had enough musical ability to recognize that Mozart’s compositions were the work of genius and his were mediocre, with is name (and music) destined to be forgotten.
Long time listener, since the begging really. As always, a great show.
Of course the title of the show will have an effect on random downloads, but it will be hard to say what would always be the best title to use for any particular show. If the content doesn’t match the title, then you will upset people eventually.
@jason I find it interesting that you said you invented a new algorithm, when in fact all you were describing was a Quadtree.
A while ago I found a good blog post touching on some spacial searching algorithms, from Quadtree, to Geohash, to Hilbert Curves.
Your problem is very similar to what MMO games face dealing with constantly moving players, and being able to show users within a certain range.
@Jacob Chapel – I thought I specified that I “re-invented” the quad tree algorithm (a realization I made after doing a Google search), but I’l go back and listen to that segment of the show to make sure.
EDIT: Okay, I just listened to the entire segment again. In the discussion, I did state that the algorithm was an “Adaptive Geo-Spatial Quad Tree”, which is obviously the name of a well-known algorithm (thus implying reinvention), but after listening to the segment again I think the reason I never got around to explaining that I had merely re-discovered the algorithm is that Justin kept interrupting me to ask about the details. But I’ll make sure to clarify this at the beginning of the next show.
@James P. McGrath – Layaway plan it is! 😉 Thanks so much for the donation and please let us know if you get any leads out of this.
@Justin – Dude, your discussion about not being happy playing music really resonated with me. I had almost exactly the same experience.
I play guitar and sing, was in two bands (one acoustic, one punk band in Pasadena, in fact), and I was unhappy the whole time because I felt my songwriting and singing weren’t “good enough.” Like you, my taste for music was far more developed than the music I was able to create. It’s one reason I focused on tech…somewhere I could create things I was proud of.
Thanks for bringing it up – a nice walk down memory lane.
BTW – Pride and Shame is a good song, sir.
@Rob, thanks for the words of encouragement. Also, thanks for the kind words about Pride & Shame. People seem to love or hate that song, and I think that’s a good sign… if people have a tepid reaction it usually means it’s.. meh.
Have you guys looked into any grants or corporate sponsorship for Catalyst laptops? Maybe grants would be too much work, but it seems like an easy sell to a tech company – a $1000 donation for them so they can write a feel good blog post about helping kids, you guys can buy a fleet of Chromebooks. Seems like a win-win.
In addition, I would add RubyMotion (37signals just released a blog post about how they used it for Basecamp iOS) and Rust to the list of “potential consultant moneymakers”.
Jason, I was really surprised about the thousand updates mentioned regarding your hackaton solution. Surely cars don’t move that fast? Would anyone notice a difference between an update on every new location report and an update once every 5 seconds or so?
It’s really impressive how the Catalyst class expands. I hope the next episode starts with an hour of shout outs after this call for donations 🙂
@Justin, regarding pursuing the music option, same here, I could not give up my tech side, although my stuff is on the experimental side. I still do it though, it’s good as a creative outlet and I come back to development with even more energy in a way. Cool episode.
Justin, regarding $$ and MVC JavaScript frameworks, the one other thing to realize is that at the end of the day the ultimate goal of any framework is to make code more maintainable, speed up development and have a great end product. If $$ does that (and clearly it does for you), then who gives a flip what you do.
The only thing to consider, however, is that there is a difference between using a framework or technology just by yourself and using something within the context of a rapidly growing startup. With just a couple other developers it is not a big deal and if you plan on growing your team slowly it doesn’t matter. The issue comes when your product hits the big time and you suddenly need to hire a dozen great developers. In that situation ends up being more about how you can attract the top talent, how quickly they can pick up your tech stack and how happy they are going to be working on it. In other words, in that context, your tech choices are less about what YOU think will work best and more about what other people will be psyched to work on. I think using $$ as opposed to Backbone/Angular/Ember/etc. in that type of situation would be difficult and create more problems than it solves.
Now, the reality though is that you are not in that crazy growth/hiring phase and who knows if you will ever get there. I agree with Jason that it is smarter to just focus on what you need to do today to get things done as fast as possible and worry tomorrow about the other stuff I mentioned. It is one of those good problems to have and it shouldn’t be a big deal converting $$ to another framework if you get to that point.
@Jeff Whelpley I couldn’t have said it better myself!
@Jeff & @ Jason – Thanks. That’s what we’re going to do! 🙂
I loved the part about the Uber Hackathon.
Jason, your adaptive matrix sounds like an octree or BSP tree?
It sounded like you will end up in trouble trying to traverse and maintain (adapt) your tree. How about using a spatial hash where all the grid cells are the same size? You can use several hashes with different cell sizes to mimic the topdown functionality of your octree. I’ve heard these work really well in games for collision detection where all the objects are the same size (cars). So you may want cell sizes that are comparable to city blocks in some ways.
Some other things to consider
– When comparing distance you only need to compare distance squared (avoid calling sqrt() on each comparison)
– You’re probably calculating distance as the crow flies, not driving distance… that’s fine, but maybe merits some second steps to optimizing dispatch
– Vehicle’s vector (direction and speed it’s driving in) can be an additional factor to how you do your hashing and query
While I assume you won’t be able to get driving distances thousands of times a second, maybe after boiling down the requests and driver locations you can end up with some candidates that you can later do more expensive checks on.
The Hungarian algorithm gives the optimal pairs from a table of possible outcomes. In a dense situation the first match for each request might not be the fastest. Maybe you need to maintain a table best matches and optimize on it on some interval.
Some reference for giggles:
Spatial Hashes
http://v7.gamedev.net/page/reference/index.html/_/technical/game-programming/spatial-hashing-r2697
Some JS
http://entitycrisis.blogspot.com/2011/07/spatial-hash-in-javascript-for-2d.html
Octree
http://www.i-programmer.info/programming/theory/1679-quadtrees-and-octrees.html
Hungarian algorithm
http://en.wikipedia.org/wiki/Hungarian_algorithm
My implementation of Hungarian – part of a game I’m making:
https://github.com/flypup/duel-of-the-devs/blob/master/app/scripts/controllers/Hungarian.js
That’s my unsolicited advice 🙂
While I listen to almost every podcast, when I don’t download, the main reason is the quality and amount of show notes, especially with relevant links.
@Jason, I agree with Stanislaw above – I’d like to know why you need to rebuild the Data structure of cars thousands of times per second? Sure once per second would be enough? The cars won’t be significantly changing position / availability in such a short timeframe.
Also, I forgot to mention before that I think the “teach your kids programming via fun game” app you are working on for Catalyst would be PERFECT for a crowd fundraising effort. It’s consumer focussed, lots of parents would want it, schools would love it. It seems to me that you could raise 50k without breaking a sweat. Even if you didn’t hit the target it would be a great way to validate the potential market for such a product.
@Jacob Chapel i did understand as “created my own algorithm”. Anyways, re-invention or creation, i really like Jason’s having a different approach to problems. I would also try to optimise on the database level like anybody else except Jason 🙂
@Rob Walch – Yeah, it turns out that the algorithm I “rediscovered” was an adaptive quad tree. Guyon and I then reimplemented our solution as a non-adaptive, 4-deep quad tree, which dramatically improved the performance. Again, we just buit all this based on intuition, mostly because we didn’t have time to do the research and it was kind of just for fun anyway. I discuss this in-depth in the most recent show, which we should have out today.
Thanks so much for the links and insight. It sounds like you’re getting into some pretty interesting math with the assignment problem and Hungarian algorithm.
BTW – the first link you provided is broken.
@James P. McGrath – Unfortunately, my explanation of the nearest drivers challenge was not very clear. You wouldn’t need to rebuild the data structure a thousand times per second, but because you could get upwards of 5 to 10 thousand location updates per second, every time the latitude and longitude change for a given car, you have to check that the car needs to be moved to a new cell, and if so, move it.
I think I agree with you about the Catalyst educational programming app being perfect for a Kickstarter campaign and it’s something I’ve definitely thought about. I’m very glad you brought it up though because you really got me thinking about it again. The only thing is that before doing so I’d like to get it developed a little further so that I understand exactly what it should be and so that I can better show off it’s potential.
@Tolga Korkunckaya – I clarified the invention / reinvention thing in the latest show which should be out today. Once again, I apologize for being unclear and not explaining it more carefully.
Also, in terms of solving the problem via database optimization, one of my Uber colleagues tried that with PostreSQL using it’s GIS module (http://wiki.postgresql.org/images/2/2e/Postgis_vincent_picavet_pgdayEU2010.pdf), but pretty much ran into a ceiling at about 500 updates per second. While that isn’t bad, we’re shooting for an order of magnitude greater than that.