How Did We Build Global Search?

In truth, I’m the wrong person to be answering this question, because I didn’t build Global Search.  Panjiva’s engineers did.

Global Search is awesome because Panjiva’s engineers are awesome.  They’re not the bragging type, but I am.  Jim Psota, Panjiva’s co-founder and CTO, has put together an extraordinary team of engineers and articulates the over-arching product vision for Panjiva, at the heart of which you’ll find Global Search.  Tim Garnett, Panjiva’s Lead Engineer and Technical Director, is the driving force behind Global Search, and Kevin Qi and Jeff Cohen round out the core Global Search team.

These guys were able to bring Global Search to life because they’re awesome — and because they had the support of so many others.  The rest of the Panjiva engineering team pitched in, of course, as did product manager Carolyn Flood and the Global Search Launch Team, which consisted of talented folks from across the Panjiva organization.

Our engineers live in the code.

But the real heroes of Global Search are our clients, who have been teaching us for years about what they need to do their jobs.  To our clients who patiently described their problems and their processes, and to the clients who gave us feedback on our earliest ideas for, and versions of, Global Search — thank you.

Ok, you’ve made it through the tech equivalent of an Academy Awards acceptance speech, and you still don’t know how we actually built Global Search.  Well, basically, we developed technology that allowed us to do in an automated way what our clients were doing in a manual way: crawling the web in search of information and then piecing it together in a way that facilitated decision-making.

We started with the companies in our database and scoured the web for publicly available information about these companies — much as general search engines do.  We paid attention to where we found this information and then looked around, on these websites, for more information about more companies involved in global trade.  When all was said and done, we found relevant information about over 6 million companies on over 7 million web pages spread across nearly 500,000 unique domains.

The next step was to give our clients an easy way to access all this information.  The Global Search interface was designed to accomplish three things:

• First, Global Search takes full advantage of all of the new data we’ve found to deliver better search results than ever before.  Our clients prefer to see companies that have matching products and that can easily be contacted.  Global Search gives priority to companies that have these characteristics and provides us with a flexible framework for incorporating even more relevant data in the years ahead.

• Second, Global Search delivers results, based on an obscene amount of data, really fast.  I can’t tell you how Tim and team made it so fast, and, frankly, I hope they won’t tell you either.  Hey, we need to have some secrets.

• Third, Global Search points you to the places around the web where all of this data originates, so you can know the source of information when determining how to use that information — and so you can easily hunt for more information.  Just as Kayak sends you out to individual airline sites, we’re excited to send you to all the places on the web where you can find more information about companies engaged in global trade.

So that’s what we’ve been up to.  It’s taken a year to get Global Search into the hands of our clients, and of course this year of work built on several years spent learning about the problems our clients faced and about how they used earlier incarnations of Panjiva.

As you can tell — after three blog posts! — we’re really proud of what we’ve built.  But, frankly, there’s so much more work to be done, so now it’s time to get back to doing it.

Why is Global Search Awesome?

You can see the official marketing propaganda here, but the punch line is that Global Search is awesome because it gives you unprecedented access to an unbelievable amount of information about companies involved in global trade — 6 million of them!

Broadly speaking, there are two classes of information that you can access with Global Search:

1. Objective information about companies involved in global trade and
2. Information that comes from the companies themselves.

For some time now, we’ve had objective information on Panjiva — information describing the track record, financial health, red flags, and certifications of hundreds of thousands of companies involved in global trade.  And, by giving our clients access to this information, we’ve made the process of finding and evaluating trading partners much easier.

Global Search builds on a base of objective information by bringing to your fingertips massive amounts of self-reported information — information provided by the companies themselves.  Of course, this self-reported information is very different from objective information, which comes from third parties.  And I wouldn’t recommend making significant business decisions based SOLELY on self-reported information.  But there are two ways in which self-reported information can be enormously helpful, particularly when combined with objective data:

1. Product photos help you figure out if a company supplies the products you need, and
2. Contact information can help you get in touch.

Global Search gives you access to over 35 million product photos, over 5 million phone numbers, and over 2 million email addresses — in addition to all the objective information that you’ve come to rely on Panjiva for.  And, of course, all of this information is intelligently organized, making it easier than ever before to find, evaluate, and make contact with companies that are right for you.  Seriously, it’s awesome.

The back-story…  Over the last few years, as we published profiles of the hundreds of thousands of companies engaged in global trade, we learned two things:

1. Our clients, global trade decision-makers, always want more information — and more kinds of information.
2. There are so many sources of information out there!

Over the last several months, the entire Panjiva team has made a concerted effort to connect our clients with more information, and more kinds of information. We announced deals with financial information provider Sinosure, social responsibility standard-setters SAI and WRAP, and illicit trade red flagger deKieffer & Horgan.

In addition to forging relationships with these information providers, we have invested in technology — the technology to take all these data sources and integrate them seamlessly. You can see the results of these efforts in the newly upgraded Panjiva Search.

But, really, we’ve only scratched the surface. onPanjiva represents the next step in our efforts to connect global trade decision-makers with the information they need. With your help, we will be identifying information providers whose data should be onPanjiva. We will move quickly to give our clients access to this valuable data — and, in so doing, give information providers the benefit of a broader audience for their data.

Lately we’ve been calling onPanjiva, “an App Store for Global Trade.” Just as the App Store connects iPhone users with applications they want, the onPanjiva program connects global trade decision-makers with the information they need.

The analogy isn’t perfect. In the first integration cycle (applications due by August 31st), we will be starting small, with a focus on quality. Information providers interested in being onPanjiva will fill out applications and provide data samples. Panjiva’s business development team, in consultation with our clients, will then choose a small number of data sources to integrate into Panjiva Search during the fourth quarter of 2009. In future cycles, we will expand the number of data sources eligible for integration.

Of course, the onPanjiva program only works with the participation of those who believe, as we do, that connecting decision-makers with information is the key to transforming global trade. If you know of an organization with valuable information, we hope that you will take 30 seconds to nominate this information provider to be onPanjiva. And if you are an information provider, we encourage you to nominate yourself.

Note: if you are the first to nominate an information provider to be onPanjiva — and that information provider ultimately publishes information onPanjiva — you will receive a year of free access to this information!

Want to learn more about the onPanjiva program? Check out http://on.panjiva.com.