Location Intelligence Conference : Location Technology & Business Intelligence

FME User Conference 2009: FME Lessens Friction in a Rough Economy

by Adena Schutzberg
June 19, 2009

Thoughts from the Top
If any company in the geospatial community holds a special set of eyeglasses to peer into that marketplace, it's Safe Software, the developer of the Feature Manipulation Engine (FME), the "glue" for many of the key implementations of geospatial technology. Safe's lead duo, President Don Murray and Vice President Dale Lutz, probably deal with more data formats, more software packages and more off-the-wall requests than many other companies combined. That gives them a unique perspective on the state of geospatial. I've been privileged to "check in" with Murray and Lutz over the last 10 years and tap into their insights. I did that again last week at the FME International User Conference, held in Whistler, British Columbia.

This year Murray and Lutz acknowledged seeing the same themes I've seen in the last 12 to 18 months: the continued explosion of Web mapping and Web services, the consumption of Web services, the drop in barriers to using geodata and geotechnology, the interest in, if not use of, the cloud, and the focus on value during this down economy.

Don Murray (left) and Dale Lutz (right).

I raised a question about the changing role of the "desktop GIS" in light of the focus on servers, services and rich, but thin, clients. They agreed that users and developers of geospatial technology need to keep a keen eye on this change, lest they find themselves out of a job or offering a product in the "wrong" space.

We spoke in some detail about the cloud and Lutz reminded me that concern about the cloud was more cultural than technological. With all the pushback seeming to be technical (security, connectivity, scalability), in reality it's a cultural shock that's afoot. And, this should come as no surprise to those who've been involved in geospatial for any length of time; the biggest challenges we face are people challenges, not software or hardware ones. Murray used the move to the centralized provision of electricity as an effective metaphor for the slow movement to the cloud. Before the advent of regional power companies, large manufacturers had their own electric plants. The switch to trusting "someone else" to provide power took some time, but ultimately became the norm.

We also revisited how best to describe what Safe provides in its FME software. Early on Safe was referred to as the "translation" company. In recent years, it took on the "extract, transform and load" (ETL) moniker from information technology. While that's still an accurate description of the desktop and server offerings, it doesn't really address the "on the fly" support and "publishing of services" now in widespread use.

What's the best way to think about what FME does for a geospatial implementation? "Glue" is a possible metaphor, but the concept that FME "lessens friction" leads me to something more along the lines of a lubricant. If, for example, prepping data for an analysis takes a few hours or a few days using manual or semi-automated methods, FME reduces that friction, reducing the time, typically from hours to minutes or days to hours. Lutz recalled, back in the 1990s, reading about how some 60% of the time/money in GIS projects involved data preparation. That's a lot of friction that can slow or even prohibit the completion of a project. What FME does, he explained, is lessen that friction by compressing the time required for data preparation. That allows solution providers to focus on providing a solution, not prepping data for that solution.

Keynote and Sessions
James Fee of RSP Architects Ltd., who blogs at Spatially Adjusted, presented the keynote titled "Removing the Barriers to Data Sharing." After reviewing the history of sharing geospatial data (from disks, to modems, to the Web, to FTP, to the Geography Network, geodata.gov, to data.gov), he offered some keys to effective data portals:

offering open formats (since the meaning of "open" may vary, "usable" may be a valuable stand-in term)
ensuring findability (he's not found great examples of this for geodata)
providing caching (for when systems go down)

He referenced recent work done for the Greater New Orleans Community Data Center built on Linux, Django and Google Maps with processing in the cloud courtesy of Safe Software and WeoGeo. Also noteworthy: a tie into cloud-based Salesforce.com to track requests for, and delivery of, geodata. For non-profits like Knowledge Works (which runs the center, as well as some for-profits), the key measure of success is documenting use of the data products; this solution does just that.

User presentations at the FME User Conference addressed quite a range of "friction problems." For example, FME powers animated visualizations of rainfall data for the South Florida Water Management District; it validates data uploaded to the Hampton Roads (VA) Sanitation District; it may, with the help of some cloud services, be the tool to rasterize and tile huge vector datasets for use in Bing Maps and other services; it powers the creation of thousands of pdfs to support infrastructure management in the field. It doesn't "do GIS" per se, but most certainly greases the wheels of GIS and location-related tasks.

Futures
Unlike many events that I attend where all the new features are announced at the opening session, the new features of FME were saved for the second day's plenary. What's coming in FME 2010?

Performance enhancements - The time for both the core processing and opening workspaces in workbench will drop. How much? The cut made available at the User Conference showed 5% and 20% enhancement respectively. Further enhancements are likely to raise those percentages before the final release.

Search added to Workbench - The ability to search for text in all the different transformers, data formats and attributes is a welcome addition. More search tools are expected for other parts of the product in the future.

Data Inspector - A new 2D and 3D visualization tool will replace the Universal Viewer. There's a nice video introduction provided by The Safe Evangelist.

New transformers - One category that will grow is generalization (smoothing complex linear features). Another that provoked some interest: a transformer that writes a message to Twitter. For now it's called the twitterer transformer, though there were discussions about whether it might have a "better" name.

FMEpedia integration with Workbench - Querying for and then downloading (courtesy of FME Server) custom transformers and others additional goodies will be available directly from the Workbench environment. Murray pulled down a custom transformer for a data format called .gps as part of a demo.

Generics and dynamics - A transformation in FME involves manipulating some source data (one or more files) and outputting some target data (one or more files). Typically, a workspace is built knowing the input and output formats and schemas (database structures). Generics and dynamics allow programmers to "leave open" the formats and schemas of one or both of these for source or target. The information to "fill in" the blanks is provided at run time. That means it's now possible to write a workspace that translates from "any" format/schema to any other format/schema.

Support for 64-bit platforms - IT staffers are likely to appreciate that addition, especially with regard to FME Server.

Murray and Lutz showed off the new functionality with some clever demos. In one, GPS tracking devices on "questionable" individuals fed data to FME Server. It, in turn, sent alerts via Twitter when individuals crossed a geofence. Phones in the audience went off as the demo ran and the points moved across the map! A second demo showed the creation of textured 3D models from basic footprints, elevation data and photos of building faces. Those data could then be used in fly-throughs and other visualizations.

FME UC by the Numbers

110 attendees
30 Safe staffers in attendance
�95 Safe staff, total
113 resellers (in 47 countries)
�50+ FME certified professionals and trainers
7,500 FME users (in 116 countries)
#1 rank of the tester transformer in the list of most used transformers

Observations
This group of users is perhaps among the most technical in the GIS arena. That seems to result in many, many plaid shirts and very few women. Most GIS conferences are far more balanced these days and I'm hopeful this community is headed in that direction. Another indication of the technical bent of attendees: As we rode gondolas up Whistler Blackcomb Mountain, exactly how the gondolas worked was as much part of the conversation as the scenery!

FME is fun. I actually sat down in the "Maps for the Masses" hands-on session and took it for a test drive. The session tackled key tools for sharing data: GeoRSS, PDF, Bing Maps, etc.

I'd never touched FME before, but I felt like a challenge and frankly I wanted to get that "over my head" feeling. Once I got a jumpstart from the teaching assistant, I had little trouble following along with the exercises. I even published my workspaces to FME Server. (It felt a lot like publishing my ArcView Maps to the Web many years ago.) My crowning achievement, however, was tracking down the typos that prevented my workspaces from running. I can see how you could be "sucked in" to playing with this huge toolbox. It's a lot like Lego building blocks; you can build whatever you want.

The visual model of programming is not for everyone. One of the more famous FME users is Peter �� Laulund. He's famous because he does his work for the Danish Ministry of the Environment National Survey and Cadastre using "map files." These files are the "scripts" that FME executes to produce output geospatial datasets, feeds and the like. Safe launched Workbench, a visual programming tool, to simplify the creation of these map files and now nearly all users depend on Workbench. But not Laulund; he carries around a thumb drive of some of his collected expertise to use in his work, and in the now annual FME Idol competition. Longtime GIS users who learn FME sometimes find Workbench a challenge. They are used to the visual feedback a GIS provides. For example, when you select features, you "see" which ones you've selected. It can take some time to make peace with the idea that the selection "goes on" behind the scenes. I'm hopeful that the more people use tools similar to Workbench (ArcGIS' ModelBuilder, Yahoo Pipes, Microsoft PopFly...), the more comfortable everyone will be with "programming."

Safe and its users like to use its software for things other than those for which it was designed. While four teams battled it out to be this year's FME Idol (a contest that requires using FME to do some GIS-y things and some not very GIS-like things), the rest of the attendees played FMEopardy (FME Jeopardy). Some Safers spent considerable time mocking up the Jeopardy board using FME Server. I, for one, was totally impressed with the outcome and had the sense that the entire team takes great pride in what they do. We also saw new works by Ulf Mansson, a user and FME artist-in-residence. His art, and that of others in this gallery, is created exclusively using FME.

Conclusion
While the geospatial industry continues to reshape itself with new platforms and new users, it seems to be creating new types of friction for FME users to alleviate. And, there's every indication the team at Safe will continue to keep its ears to the ground to keep up with user demand. One key indicator: The bulletin boards set up outside the session rooms for "suggestions" were completely full of pinned-on messages at the end of each day of the conference.

More about this author...