I know from long stints in leading a couple of companies that you have to be prepared to take some flak for what you believe in. It looks like this is one of those weeks. Here is the latest post.

While I see Sebastian has already taken some blame for this section of my post, I will note that I would not have included it if I did not agree with what it said. I put my name only on those things for which I’m willing to stand by (unless convinced I’m wrong, in which case I would apologize and retract, neither of which I’m going to do here).

First let me point out that while I’m a relative newcomer to the FLOSS world, I believe I’ve demonstrated long, strong support for open standards, which our announcement that started all of this is based. My participation in the National Information Standards Organization (NISO) for which I’m currently the Immediate Past Chair is now described using the word decade plus. My participation in the FEDORA community now exceeds 4 years and I serve on the Fedora Advisory Group. Using FEDORA, I lead a project to develop a product that is in use around the world. The FEDORA user list is a bit more sizable as a result. I do papers, podcasts and presentations concerning Open Source Software and the reasons why it is important. I’m very proud of my contribution to the library community via open standards and, while relatively new to the FLOSS world, I’m equally proud of the work I’ve accomplished here thus far. I’ll also note that I’m putting my money where my mouth is, because I’ve founded and solely financed a company to support FLOSS applications. So, I believe I clear the “honest broker” bar described in the post.

The post takes me to task for “offending” the FLOSS community. Really? My movement to this community, from the proprietary software community, has been noted and appreciated in various blogs as an positive development for the FLOSS community. See this post and this post as examples. If our recent announcement and my recent post have somehow undone that goodwill, then I will gladly admit to not being a hardliner or zealot. Because surely only those who would view these actions in that light would describe it in that way.

Am I commercializing FLOSS? You bet. At this past ALA, my company announced CrOSS, a program for specifically commercializing open source applications (i.e. taking them out of academic, spare time settings) and converting them into supported, marketed and readily accessible applications for ALL sizes and types of libraries (not just those that have IT staff in them). Is this wrong or contrary to the goals of FLOSS? Perhaps you could, in a very narrow and negative way, view it purely as commercialization. I personally see it as adding substantial value to those applications and helping them build a much larger and more sustainable communities in which they will “live long and prosper.” Surely that is a goal shared with FLOSS developers? I thought it was, but perhaps I’ve misunderstood? I’ll note that at ALA, the announcement of the CrOSS program resulted in numerous, positive meetings with academic institutions looking to learn more about the program and how it could be adopted to promote their FLOSS applications. I think that alone is endorsement that people understand that for larger market penetration and acceptance of FLOSS, libraries want and need the added value services CrOSS supplies. FLOSS can be portrayed as widely accepted and used, but the surveys such as Marshall Breedings just completed survey, and others, show that FLOSS is still very, very much on the earliest portion of the technology adoption climb. Breeding’s survey shows that of those libraries planning to replace their ILS, at best, only around 5% are currently considering open source. That says to me there is much left to be done. (Yes, we all know they are in fact, using OSS apps like Apache, Linux and others, at far higher percentages, but I’m talking library specific applications.) It’s making headway, it’s clearly growing, but to say it’s mainstream would be a gross misstatement of reality. My actions and those of my company are designed to make that number grow much larger and much faster.

As for interpreting the statements about open source being “academic roots” and a “sexy word” implying that FLOSS isn’t prevalent or can’t coexist with commercialization, I guess I now have a far better appreciation of how the current round of presidential nominee politicians must feel. That is NOT what I was conveying at all and is, I believe, a very selective spin on the statement. I’ll point out that the statement was in reply to the original post that started all of this, where the post was taking issue with our calling OpenTranslators “open” when in fact it combined open standards with proprietary technology. Our reply was designed to point out that, in our minds, for FLOSS to move beyond it’s pure, zealot roots and to deliver TOTAL working solutions to people’s desktops, there would be times we would need to combine both proprietary and open source technologies. In fact, as this post and this post about OpenTranslators have pointed out with this new offering, when coupled with many open source solutions, libraries can now provide new (and frequently) matching or exceeding levels of functionality as those obtained with pure proprietary solutions. Again, I thought this was in line with the goals of FLOSS (maybe I haven’t been fully assimilated yet?). If and when, pure open source translators emerge, we’ll be glad to adopt them. Until then, we’re in the business of advocating and developing SOLUTIONS using open source software and standards wherever and whenever possible.

Does that reflect 1998? I don’t think so. Our feet are firmly planted in 2008, dealing with the realities of a marketplace starting to transition. I think we’d all be better served not by drawing up fences and attacking, but by reaching across lines and building alliances, friendships and partnerships while looking for solutions that benefit us all rather than just a few. Surely life in this country has taught us that much recently? It certainly describes what I’ve been trying to do. If that makes me delusional, so be it.

We found this post on the “Federated Search Blog” about our recent announcement at ALA concerning OpenTranslators. It raises some very important questions, conveys a misunderstanding and possibly some philosophical differences, but the important point is that it provides an opportunity to discuss why we believe OpenTranslators are important.

CARE Affiliates(TM) believes, as does the writer of the Blog above, that the OpenTranslators amounts to something “That… is pretty significant” in the federated searching world. What we did, was to bring together a total of three companies, each contributing expertise, products and ideas to create a new way for users to have access to thousands and thousands of databases, some of which support proprietary API’s of their own, but most of which support no interface at all beyond the end-user HTML interface. These have to be parsed and interpreted to support metasearching. Some represent freely available content, but most are subscription databases of the sort used in the hundreds by a typical library to meet patron needs. So bottom line, what does that mean? It means that the search interface used by your library and thus, your users, is yet further decoupled from the content underneath.

First, a little history. Years ago (and it is still true in some cases) if you wanted to search certain content, you could only do it using the interface provided by the company that assembled that content. The interface and content were tightly coupled. This was very frustrating for users as it meant that for each resource they wanted to use, they likely had to learn a new search interface.

Federated search tools eliminated that problem and allowed a common search interface layer to many resources. This was better for end users who now only had to learn one search interface; that provided by their federated search vendor. The downside was that in order to search to all those content resources, there had to be a translator or connector developed that could talk the language of the original content providers search interface and translate that to the language of the new federated search providers interface. So in the end, the vendors’ translators/connectors were tightly coupled to the vendors’ search interface. The search interface had moved up a layer in the stack, and you as the customer had more options/choices, but we saw the possibility to provide even more choices.

Our colleagues at Index Data(TM) had built software to take SRU/SRW/Z39.50 searches and translate them for use with translators/connectors. Couple that with the fact that we had signed a deal with WebFeat(TM) in August of 2007 that allowed us to sell their translators with our products. We did (and do) this with the metasearch/federated search application called MasterKey.

While we thought it was very progressive of WebFeat to allow us to couple an OSS based metasearch/federated search solution to their translators, all three companies had a vision of carrying the concept yet further. Why not take the Index Data SRU/SRW/Z39.50 translators, couple them with WebFeat’s translators and allow the end user to pick the metasearch interface of their choice? It could be their existing OPAC. It could be anything that could talk SRU/SRW/Z39.50; a large and growing selection of software products, both open and proprietary; it could even include homegrown applications and mashups. It would mean that we had yet further decoupled content from the search interface and were putting total control of the interface selected into the hands of the customers. That was the basis of the OpenTranslators announcement.

Now, with that history and background, let’s answer some of the questions/points raised in the Blog post that started this conversation.

First, we want to offer a correction to the statement that “WebFeat has a large number of translators (connectors to SRU/SRW/Z39.50 databases).” Actually, Webfeat has translators to thousands of databases that are licensed, proprietary content, the vast majority of which are NOT accessible through SRU/SRW/Z39.50. Only with the announcement of OpenTranslators does that become true.

“1. How “open” are these OpenTranslators?” Our response is that what is “Open” is that we’ve taken open standards and enabled access to the translators. As a result this allows you to leverage existing software applications, programmer’s toolkits, as well as writing your own applications to interact with the translators, through SRU/SRW/Z39.50. Yes, the translators themselves incorporate proprietary technology and will continue to do so. Anyone who has tried to develop a translator will know it is a complicated and demanding process, and that the final product requires constant monitoring and maintenance because back-end interfaces are subject to change at any moment.  Consequently, it is an application that is ideally suited to operate as a hosted service, where revenues is earned from keeping the translators up-to-date and backed bu sufficient CPU cycles to meet requirements.

It is also important to note that we’ve always said that while we believe in OSS, we also know and understand there will be times where we will need to marry OSS and proprietary technologies to provide a total solution. So, bottom line, what we’ve done, is to expose proprietary technology through open, standards-based protocols as a way for end users to pick the metasearch/federated search interface of their choice and still have access to vast amounts of content. It allows developers to focus more attention on interfaces and usability, and less on sorting out how to access a given database. It also dramatically opens the field of search-oriented projects that may be undertaken either by libraries or by startup companies. We believe that this is a dramatic step and one that qualifies as “open”. (We’ll expand on this later in this posting).

“2. How does access to content work when authentication is involved if the connectors to the SRU/SWR/Z39.50 database are WebFeat connectors and there’s a CARE/Index Data gateway in between?” When the service is configured for a customer, WebFeat handles authentication against each database on behalf of the customer. All you need to provide to the OpenTranslator is a customer ID, which is provided to you when you sign on.

“3. What will access cost? What is the licensing model?” As noted elsewhere in the original blog posting, this is a hosted service, so what customers pay is an annual subscription to use both the CARE/ID OpenTranslators and the WebFeat translators. The cost is based on the number of translators used and, as with most such services, the more you buy, the cheaper they get. One yearly cost covers both subscriptions. Interested parties should contact CARE Affiliates for a quote.

“4. Are there really 10,000 databases accessible from this service as advertised in the press release?” WebFeat states that they have 9,000+ databases in their translator library and Index Data provides many thousands more pure Z39.50 connectors. Together, it totals over 10,000. Do some of those databases come from the same content providers? Sure. Like every industry, once you get good at something, you tend to acquire similar companies and the result has been that there is no longer one company per database, nor should there be. Cost efficiencies result from using the same infrastructure to supply access to many different kinds of content. This helps keep costs down for the end users.

“5. Who is hosting access? Is there a plan to handle scaling of the service if it becomes real popular?” Both CARE/Index Data and WebFeat use hosting centers. These kinds of companies offer real benefits for these types of services because they offer redundant processors, telecommunication lines, mirrored disks and instant fail-overs. Plus, they can add additional capacity, usually within hours of a request. So, are we well positioned to scale this service? Absolutely, we’ve planned for it and we’re counting on it.

So, here is a reason why we think OpenTranslators are so important. For OSS to go beyond being an academic practice; beyond being an activity practiced in basements in people’s spare time; and finally beyond being just the sexy word that it is at the moment, then it’s really important that we work out ways to make OSS (and for that matter, open standards, which is really a variation on the theme) coexist with, and indeed support commercial activities. Ultimately that is how things will get done and how usable products will result.

We’re looking for as many areas of synergy as we can, rather than painting OSS as being a threat or even necessarily a total alternative to commercial players. We believe it *should* be viewed as a threat to the stale business practices of many current vendors, but it shouldn’t be viewed as being in opposition to commercial endeavors. OpenTranslators make important headway not only because of what it does for metasearch/federated searching, but also as an example of how total solutions can be assembled using best-of-breed products and companies to deliver what customers want, directly to their desktops. We are crossing new ground that will lead to better products and services for everyone. That is what OpenTranslators do and why they are important.

There are moments in time, when you know things have changed, not just a little, but a lot. Sometimes, they’re political (in my memory, things like JFK’s election and assassination, Robert Kennedy and Martin Luther King’s assassinations), sometimes global (landing on the moon or September 11th), sometimes personal (you and your spouse have a child), and sometimes they’re professional. I’ve just returned home with this feeling about the profession fresh in my mind now that ALA is over and now that I’ve had a moment to reflect on what I saw, heard and participated in with regard to the library automation landscape. There is a shift underway in library automation. It is major and I won’t be surprised if it becomes seismic in the end. Let me explain why I think that is the case.

When we were planning for this conference, we had a feeling this would be one where people would want to know about open source software and solutions. We felt we had an opportunity to provide information — a lot of information, and people would seek it out and readily absorb it and make decisions using it. Based on that feeling, we had planned a number of seminars in our booth and our colleagues at LibLime did the same. These seminars were focused on providing, via library professionals and colleagues, descriptions of what could be done with open source software, the possibilities, probabilities and processes. We launched the seminars not knowing for sure if we were taking the pulse right, but definitely thinking we were. We walked into the conference wondering if we would be talking to a few people or, would it be standing room only? It did not take long for us to get the answer. After the first one, we knew we were going to be dealing with a lot of standing room only sessions. It didn’t matter if the session was on integrated library systems, OPAC replacements or repositories, people seized the opportunities provided to hear, to learn and to discuss with colleagues, what was happening. (See the pictures below). They were standing in the aisles, in any open spot, leaning in, listening intently, watching closely and nodding approvingly. They liked what they were seeing, liked what they were hearing, wanting to be part of something that they too could feel growing, embracing them and providing answers to a set of problems that had been dumped on them without their consent, by those whose needs concerns are expressed solely through spreadsheets. The customers of those firms were saying “Enough, it’s time to look at the options.” And open source is one of those options, one where the freedom, the control, the focus on service rather than dollars and the better return on investment for the librarian, rather than the vendor is presenting an overwhelming set of reasons to consider new options. The number of people leaving business cards requesting follow-up is very impressive.

A friend and colleague, who I deeply respect, said to me: “The story of this conference, with regard to automation, would be to have a picture of the open source vendor booth(s) at last ALA and then to compare it to the picture of the same thing at this ALA.” Well, fortunately, I have a few of those pictures and I enclose them below. What is the saying? “A picture is worth a thousand words??” Well, in that light, I’ll leave it to these pictures to finish the majority of this post…

This was the open source booth at the last show, six months ago. Three vendors sharing a 10×20 booth:

This is from ALA, Mid-Winter 2008, just completed, six months later. Note the people standing in the aisles to hear about Open Source Software and Solutions.

Compare our presence below (this is from setup, before the doors opened for people to surge in) to that in the photo from Mid-Winter (two pictures above). Two 20×20 booths and we could have used more…