Archive for the ‘Databases’ Category

File Under: Databases, Web Services
Apr
7
2011

OpenStreetBlock Gives Geodata the Human Touch

By

Location-based web services are all the rage right now, but for most of us the actual geographic location isn’t very interesting — do you know where “40.737813,-73.997887″ is off the top of your head? No? How about “West 14th Street bet. 6th Ave. and 7th Ave?”

For the geographic web to become useful geodata has to be converted into something humans actually understand. Enter OpenStreetBlock.

OpenStreetBlock is a new web service that takes geographic coordinates (latitude/longitude pairs) and turns them into an actual city block description. The result is textual information which, in many cases, will be even more meaningful to your users than the ubiquitous pin on a map.

If you’d like to play around with a sampling of data from New York, head over to OpenStreetBlock and try out the New York demos.

If you’ve ever wanted to build your own version of EveryBlock — which pinpoints events, news stories and public data at the city-block level — OpenStreetBlock will go a long way toward getting you there. So long as you can pull geo coordinates out of your source data, OpenStreetBlock can turn that into more meaningful information.

Under the hood OpenStreetBlock relies on OpenStreetMap data and uses PHP in conjunction with a geographic database to turn your coordinates into block descriptions.

As cool as OpenStreetBlock is, getting it up and running on your own site will require a bit of work. Luckily, there are some good tutorials available that will walk you through the process of installing and setting up many of the prerequisites like PostgreSQL and PostGIS (I’ll assume you already have an Apache server with PHP installed).

To get started with OpenStreetBlock, grab the code from GitHub. The next thing you’ll need is a PostgreSQL database with all the PostGIS tools installed. Luckily those are also prerequisites for GeoDjango, so head over to the GeoDjango installation page, skip the Django-specific parts and just follow the Postgres and PostGIS installation instructions.

Next you’ll need to download Osmosis and Osm2pgsql to convert OpenStreetMap data into something Postgres can handle. Head over to OpenStreetMap, zoom into an area you’d like to query with OpenStreetBlock and then choose “export.” Select the OpenStreetMap XML Data option and save the file.

From there you can check out the guide to importing the OpenStreetMap XML Data in the OpenStreetBlock read me.

See Also:

Tags: ,
File Under: Databases
Sep
22
2010

Open Data’s Access Problem, and How to Solve it

By

The recent Gov 2.0 summit in Washington D.C. saw several promising new announcements which will help government agencies share code and best practices for making public data available to developers.

The idea behind new projects like Challenge.gov, the FCC’s new developer tools and the Civic Commons is that by giving developers access to data previously stored in dusty filing cabinets, they can create tools to give ordinary citizens greater access to that data.

Unfortunately, not everything open data project leads to good things. It is critical that if open data is made available on the web, it must be accompanied by some effort to ensure everyone can access it.

We’ve seen an explosion in creative hacks that use this newly available data to provide excellent online resources. Public data sites like EveryBlock, or the Sunlight Foundation’s Design for America contest have highlighted some of the amazing ways open data can make our lives better. Whether it’s finding out crime stats, real estate values, health hazards and business license statuses in your neighborhood, or visualizing how the government is spending your tax dollars through innovative maps, open data and what you can do with it is the current hotness among web developers.

Most of the benefits are close to home — in the U.S., just about everyone has access to online government resources thanks to web-enabled computers in free public libraries.

But extend that argument to the rest of the world and the number of people that really have access to the data drops significantly. If you don’t have an easy way to get online, you can’t benefit from open data.

Michael Gurstein, Executive Director of the Center for Community Informatics Research, recently highlighted some of the problems with open data accessibility.

Gurstein points out a number of assumptions about open data that are often overlooked by those most enthusiastic about making such data publicly available.

Worse, he shows how such data can be used against you.

Continue Reading “Open Data’s Access Problem, and How to Solve it” »

Tags: , , , ,
File Under: Databases, Other
Jul
23
2010

Big Data in the Deep Freeze: John Jacobsen of IceCube

By

John Jacobsen works for the IceCube telescope project, the world’s largest neutrino detector, located at the South Pole. The project’s mission is to search for the radioactive sub-atomic particles that have been generated by violent astrophysical events: “exploding stars, gamma ray bursts, and cataclysmic phenomena involving black holes and neutron stars,” according to the project website.

Jacobsen is one of the people in charge of handling the massive amounts of data collected by IceCube. In the video, shot this week at the O’Reilly OSCON 2010 conference in Portland, Oregon, John explains how they collect a terabyte of raw data per hour, then send everything to IceCube’s remote research and backup facilities using a finicky satellite hook-up.

Antarctica is one of the least accommodating places on Earth to perform scientific research with computers. It’s the driest spot on the planet — atmospheric humidity hovers around zero — and bursts of static electricity threaten the integrity of IceCube’s data stores. The lack of humidity causes the server clusters’ cooling systems to break down. And if something fails, a spare might take six months to arrive.

Tags: , , ,
File Under: Databases, Visual Design
Mar
29
2010

Sunlight Labs Offering $5K for Best Government Data Mashups

By

designforamerica

Artists, web developers and data visualization geniuses, here’s a chance to strut your stuff, serve your country and win some serious money in the process.

Sunlight Foundation, a non-profit organization that provides tools to make government data more transparent, has announced a new contest called Design for America. Billed as a “design and data visualization extravaganza,” Sunlight is encouraging the public to create and publish data visualizations that help make complex government data easier for people to digest and interact with.

There are several different categories open for submission, including: visualizations of Recovery.gov data that shows how the stimulus money is being spent, visualizations showing how a bill becomes a law, a redesign of a .gov website, and a redesign of any government form. Top prize in each category is a cool $5,000.

Creations can be in any form — a website, a game, a poster, a sculpture, whatever — though we suspect most of the entries will be either posters or interactive Flash graphics.

The contest is being run by Sunlight Labs, the skunkworks wing of the larger Sunlight Foundation. The Sunlight group spends most of its energy collecting government data, organizing it into publicly accessible databases, then creating tools that make it easier for ordinary people to access that data. The non-profit works with organizations like OpenCongress, MapLight, FollowTheMoney and USASpending.gov. Sunlight also maintains a list of APIs developers can use to access the data.

The Design for America contest encourages participants to sift through the vast datasets available from all of these organizations, as well as the datasets maintained by Sunlight Foundation and any raw government data that’s available. As the Sunlight Labs blog says, the goal of the contest is to “tell interesting stories” that go beyond what can be an overwhelming amount of unfiltered data.

Visualizations can be in any medium, not just the web, so if you’re a video or infographic specialist, you can still enter the contest. The main criteria for judging are the visual quality of the artwork and how well the underlying information is conveyed.

Continue Reading “Sunlight Labs Offering $5K for Best Government Data Mashups” »

Tags: , ,
File Under: Databases, Multimedia
Feb
15
2010

Build a Website With Flash and MySQL – Lesson 1

By
Overview

There’s always been somewhat of a disconnect between designer-types and backend-types. And we’re not just talking about hairstyles, eyewear, and the contents of their bookshelves — designers and programmers approach problems in different ways, and their individual plans of attack don’t always jibe. Of course, you want your site to be sleek, fast, and bleeding-edge, but are you willing to compromise stability, scalability, and compatibility just so the users can ooh and aah at a few pretty pictures? Web design has always been a balancing act, and the ideologies of each camp often clash.

But Webmonkey’s here to say that it doesn’t have to be that way. It is possible to create a site that has a lightweight Flash frontend, a dreamy interface, and a scalable, secure, and dynamic backend.

Webmonkey Scott has found the balance between eye-catching design and backend database wizardry. Lucky for us, he’s agreed to share this knowledge in a two-day tutorial. He’s even put together a hands-on project that shows you how to build an easy-to-update blog using Flash. That’s right, a Flash-based blog — it sounds a little loony, but this blog pulls content from a MySQL database and feeds it into the dynamic Flash frontend using a few lines of PHP code.

In day one, Scott talks you through the construction of an open source MySQL database using both the phpMyAdmin tool and mysql on the command line. You’ll learn how to add blog entries to the database and then you’ll learn how to run queries in PHP. Topics such as basic database organization and the behavior of variables are also covered in this lesson.

Day two shows you how to design and build the blog’s Flash frontend. You’ll use common Flash ActionScript objects to pull the content from your MySQL database to the different areas of your blog’s user interface. You’ll also learn some common workarounds to keep all of your content flowing smoothly.

Whether you are designer or a programmer, it’s time to roll up those sleeves and get ready to see how the other half lives. Even if those of you who don’t feel a particular loyalty to either side of the fence will still discover that there’s plenty of hands-on knowledge to be gained.

We’re not promising that designers will sell their $500 pencil sharpeners, start marrying UNIX geeks, and honeymooning in Cancun before breeding programmer/designer children, but we will insist that they sit in a room alone together until they can build a useful website that everyone can agree on. Hey, stranger things have happened. Like Carrot Top.

Continue Reading “Build a Website With Flash and MySQL – Lesson 1″ »
Tags: , ,
File Under: Databases, Programming
Feb
15
2010

Build a Website With Flash and MySQL – Lesson 2

By

In Build a Website With Flash and MySQL – Lesson 1, we successfully created a MySQL database, filled it with blog entries, and learned how to query it. Now we’re going to move on to the fun stuff:creating a Flash container to display our blog entries as we pull them out of the database. I’m going to help you build something along the lines of what you’ll encounter at my own Flash blog site, Luxagraf.

Fire up Flash and create a new document. The first thing we need is a nice big text field to display our entries. You could create a text field in ActionScript if you like, using the createTextField() method. I don’t know about you, but my stomach for code is still full from yesterday, so I’ve just drawn a text field using the cursor tool and given it an instance name of entries_txt. In the Properties Inspector, set the text to be dynamic, multi-line, and HTML formatted. For safety’s sake, select the character option and click “embed all characters.” Flash has some issues with dynamic textfields that don’t have embedded characters, especially if you put your text under a mask. Continue Reading “Build a Website With Flash and MySQL – Lesson 2″ »

Tags: ,
File Under: Databases
Feb
15
2010

PHP and MySQL Tutorial – Lesson 2

By

In this lesson, we’re going to dive right in and create some simple yet useful pages using PHP and MySQL. Let’s start by displaying the database we created yesterday in Lesson 1, but with a little more panache.

Continue Reading “PHP and MySQL Tutorial – Lesson 2″ »

Tags: ,
File Under: Databases, Programming
Feb
15
2010

PHP and MySQL Tutorial – Lesson 3

By

Welcome to the third and final lesson for this tutorial. If you’ve gone through Lesson 1 and Lesson 2, you already know the essentials for installing and writing useful scripts with MySQL and PHP. We’re going to look at some useful PHP functions that should make your life a lot easier. First, let’s look at include files.

We all know the basics of includes, right? Contents of an external file are referenced and imported into the main file. It’s pretty easy:You call a file and it’s included. When we do this in PHP there are two functions we need to talk about:include() and require(). The difference between these two functions is subtle but important, so let’s take a closer look. The require() function works in a XSSI-like way; files are included as part of the original document as soon as that file is parsed, regardless of its location in the script. So if you decide to place a require() function inside a conditional loop, the external file will be included even if that part of the conditional loop is false.


Continue Reading “PHP and MySQL Tutorial – Lesson 3″ »
Tags: , ,
File Under: Databases, Programming
Feb
15
2010

Cake and Trax for Beginners

By

In my last article, Building With Ajax and Rails I made a faintly disparaging joke about some new web frameworks that have been created in fond imitation of Rails. I got a lot of feedback about that joke. I’m not allowed to comment here about the pending lawsuits, but I would ask that the drive-by eggings of my house and threats to my family please cease. (They’ve been relocated to a secret Webmonkey farm anyway.)

Today we’re going to take a look at a couple of those frameworks for PHP:Trax and Cake. Both attempt to bring the quick, easy, helpful tools and easily understood, easily maintained structure of Rails to PHP — a boon to web developers who know PHP and perhaps have some keeper code in that language, but can’t resist the Rails buzz. Both Trax and Cake use the same model-view-controller pattern and Active Record ways of addressing data that Rails does. Makes one curious, no? I don’t have time to get deeply into them today, but both stress “rapid development,” so let’s see if I, your average not-too-bright web developer, can get a little app off the ground before the end of this article.

Continue Reading “Cake and Trax for Beginners” »
Tags: ,
File Under: Databases
Feb
15
2010

Manage Transactions in MySQL

By

Overview

In this tell-all tutorial, Jay Greenspan, author of MySQL Weekend Crash Course and co-author of MySQL/PHP Database Applications, starts with a tour of the basics: He answers the age-old Q: “What’s the big deal with Transactions?”; investigates the four properties that a database must have to be considered transaction-capable; takes a closer look at locking mechanisms; and finishes up with a look at MyISAM tables, the lesser cousin of fully transaction-capable tables.

Once you have a taste of the limitations of MyISAM tables, you’ll be hungry for the real deal. In Lesson 2, Jay satiates that hunger with a thorough introduction to MySQL’s different transactional table types: BDB, Gemini, and InnoDB.

Get started: Lesson 1

Tags: ,
« OLDER POSTS