Tag Archives: Meta-directory

Enter the Migrator

One common business case we get is to migrate from various directory servers to AD. This is usually an issue of per user license cost but lower maintenance is also a factor. Companies are realizing that since they are maintaining AD anyway, why pay for and maintain other multiple directory servers as well? For employee accounts it usually doesn’t make sense to have the same account in two places and need additional processes just to keep them in sync.

There are several ways you can migrate directories. You could use a one-time import/export, a Metadirectory, or a provisioning system, but these approaches have several key drawbacks. One issue is that in most cases you can’t migrate the user passwords. Another issue that the migration may require custom attributes to be added to AD (try getting your AD team to agree to that).

But the biggest issue is that these directories exist for a reason. There are client apps, sometimes tens or hundreds, which rely on the information in the old directories. Most home grown apps written for one directory won’t be able to switch over to AD without extensive rewriting. Even commercial apps that support AD may require significant and disruptive configuration changes.

Enter the Migrator (obscure Disney reference intended)

A virtual directory can be your Migrator. The solution is to standup a virtual directory that merges your AD with the old directory into a single view that emulates the old directory. Run both directories side by side while migrating the accounts. When a password change is made the virtual directory can update both AD and the old directory with the new value, so after running side-by-side long enough, most of the passwords will have been migrated. Eventually the old directory can be retired.

This approach has two main advantages:

  • no changes need to be made to the client applications
  • no schema changes need to be made to AD.

There is a good white paper that covers this in detail on the OptimalIdM web site (no registration required).


Phillips versus flat-head, for real this time

A while back I wrote this post comparing the argument between Virtual Directories and Meta-directories to an argument comparing Phillips and flat-head screw drivers. I forgot about it until I noticed something interesting. Somehow a lot of readers found there way to the post from a thread comparing Java and C#. But another group of readers found arrived by searching for, oddly enough, information about Phillips versus flat-head screws.

Now I have no more interest in getting involved in the Java versus C# language debate than I am in the Virtual Directory versus Meta-directory debate.

But Phillips versus Flat-head screws? Boy have I got some opinions on that.

If you have to work with existing screws your choice has already made for you (just as if you join a project in progress you seldom get to choose between Java and C#). But if you are starting a project from scratch, you not only have to choose between Phillips and Flat-head, there is also Torx (the commercial name for hexlobular internally driven screws), square, hex, Allen (internal hex), one-way-flat-head, spline drive, etc. Just as you might consider more choices than Phillips and flat-head, you might also consider a myriad of programming languages in addition to Java and C#.

But some are clearly superior to others in certain aspects. Flat-head screws have more driving power than Phillips head screws (Phillips head screws are designed to cam-out to prevent over tightening, an intentional design feature). But flat-head screws are much harder to drive by hand due to tool slippage. This is an interesting analogy to  the ease of development of C++ versus both Java and C#.

Allen and hex head screws have even more driving power than flat-head screws and are easier to user, but suffer from the limitation of needing have the exact size tool to fit the specific head size, whereas flat-head screws can accommodate a wide variety of tool sizes. This is similar to how scripting languages are often limited to use in a specific framework.

Wait, am I still talking about screws?

Directories, Virtual-Directories, and Vendor Independence

Nishant Kaushik has some interesting thoughts about the Virtual-Directory vs Meta-Directory debate. He makes some good points, and has this to say about vendor lock-in:

Well, from the standpoint of a deployer/implementer, I can certainly understand the attraction of the above. But as a product architect and technologist, all I can say is “No, No, No”. Why would we want to tie ourselves into a non-competitive, no-way-out scenario that we see repeated over and over in the OS and Mobile Provider worlds? Choice is necessary, nay vital, to innovation and growth. The minute you lock yourself into a single provider model, you are doomed to forever be curtailed by what that provider dictates. Virtual Directory provides a nice abstraction that frees you from having to make these very decisions on which directory to support (something LDAP was supposed to do, but didn’t).

And how are more applications supporting AD anyway? A lot of that has to do with the emergence of Virtual Directory solutions. A number of applications in the Oracle stable today claim to support AD as the identity store. The mechanism for all these is moving to Virtual Directory NOT because Oracle has a Virtual Directory product, but because maintaining adapters/connectors/plugins and what have you for all LDAP variants is a colossal nightmare.

So this is an interesting question. Does moving to a virtual-directory architecture free the customer from vendor lock-in, or does it lock the customer to their choice of virtual-directory providers? Or put another way, could you deploy a set of these products and realistically swap out OVD and replace it with the virtual-directory from OptimalIdM or Red Hat? If technically possible, would Oracle support it? I suspect I know the answer to that last one.

BTW, having written code that supports multiple LDAP vendors at four different companies and three different programming languages, it’s really not all that difficult. The real power in virtual-directories is the ability to consolidate data from disparate sources, not abstracting the vendor for a single directory.

Directory vs Virtual-Directory

Now here is a twist. Instead of the traditional Meta-directory vs Virtual-Directory debate, we may have a Directory vs Virtual-Directory debate. Alex Karasulu of the ApacheDS project left this interesting comment on my post about the Red Hat Directory Server:

One last thing. The VD implementations of today like Penrose, are just hacks without a formal computational basis to them. People trying to get a product to market rapidly to sell a company. We intend to enable virtualization eventually with a solid footing in the LDAP administrative model using this concept of a view. Views, as well as triggers/SPs will enable new ways to easily solve the problems encountered in the identity space. As a teaser just think what could be done in the provisioning space if AD supported triggers? Real technology will yield solid reliable solutions instead of these band aids we’re seeing during this identity gold rush.

Pretty strong words, but the idea behind them is very interesting. Instead of building out a virtual directory, add the same capability by using views and triggers in the underlying DB. Personally I’m skeptical that this could achieve the same level of flexibility for talking to disparate data sources that you have with today’s virtual-directories. But hey, if the ApacheDS guys think they can pull it off, more power to them.

It does seem that this solution would require all the disparate data to be written the same DB that is back-ending the ApacheDS. Perhaps a Meta-directory could be used for this (irony intentional).

It is, after all, just a tool

Matt Flynn has some interesting thoughts on the Virtual Directory vs Meta-Directory debate:

I love virtual directory technology as much as the next guy (Hi Mark), but claiming that any technology is superior to another without a discussion of the specific requirements being met just doesn’t seem to make sense. Companies, departments, and projects within departments have different needs.

I’ve said it before. They’re just tools. So, when James McGovern asks what the role of virtual directory should be, I don’t have an answer. There is no should in this discussion. Ian Yip had a similar pragmatic answer. And Nishant echoed with “the mantra should always be to choose the right tool that solves your problems”. Exactly.

Yes, exactly. This should be about choosing the right tool for the job, not debating philosophies.

A good point, taken too far

Mark Wilcox has a very informative post where he answers some question about directories and virtual directories. But he does go a bit far when he says:

I don’t cover pricing because there is a lot of negotiation that goes into it.

Anyway ADAM is a meta-directory solution and as with any meta-directory solution – the license cost is the smallest part of the total cost.

Meta-directory means you must synchronize data, backup that data, secure that data, make it highly available by replicating that data, monitor that all of that is working and assign someone to manage that data.

I shouldn’t have to remind you – that you’re already doing that with your existing sources – why would anyone want to redo all of that work? Virtualization allows you to increase your return on existing investments (ROEI) because you can leverage what you already have done in new ways.

Even if the license was free (I mean you could do what ADAM does with OpenLDAP or Fedora directory and they are free) – you can see that license cost is not the true total cost.

I can see where he is going with this; ADAM is often used in conjunction with ILM. But not always, and I’m not sure it’s even the majority. I have seen plenty of installations of ADAM that had nothing to do with meta-directories. ADAM is an LDAP directory server, end of story. If you label that a meta-directory solution, then so too would be Sun DS, OpenLDAP, and (ironically) OID.

I have failed to communicate

Clayton Donley of Oracle makes some excellent points here about my “Elephant in the Room” post.  But I have apparently done a poor job in communicating the whole point of the post. In his post:

What I think his post misses is the fact that most LDAP access in most applications is poorly written, even when using ADSI or ADO to talk natively to Active Directory. I can’t count the number of virtual directory deployments that we’ve sold to help customers in environments that were nearly 100% Microsoft (ADO/ADSI-enabled apps talking to Microsoft AD). Many of these deployments were to get around bad schema assumptions, others were to get around topology issues or forest boundary issues.

While we sell virtual directory technology, we hate making our customers pay money to solve such tactical issues. We want to be layering on higher-order value.

So when Phil Hunt or others talk about the Liberty IGF project, what they’re really saying is that we want a better way to give application developers a way to code something in a way they understand and can do well rather than a native access protocol that requires specialization. So while LDAP isn’t going away and everything from virtual directories to identity buses will need to support native access over LDAP to be successful, not looking at what developers are learning and using every day would be a mistake.

Keep in mind that developers must integrate with a LOT of technologies to build an enterprise application or portal. For example, a portal may be integrating with HR, CRM, and ERP systems. That integration is increasingly happening via web services. Giving these developers a mandate to use a completely different type of technology to integrate identity will only make identity more specialized and less standardized and understood over time. That is a recipe for disaster.

I did not mean to imply LDAP was a better choice that Liberty IGF. I was in fact the BMC rep to Liberty TEG and am very supportive of their work. I also agree there are a lot of problems with LDAP and how developers use it.

But having been involved at some level in the standardization efforts of DAML, XRPM, DSML v2, SPML, SAML, WSDM, OATH, Liberty, WS-Trust, WS-SecureConversation, and WS-Federation. I have spent a lot of time working on identity service standards and developing implementations of those standards at several different companies.

But the hardest thing is getting adoption of these standards. The point of my post was not to suggest that standards for identity services other than LDAP aren’t a good thing. The point was that to drive adoption you have to accept the reality that AD and other LDAPs have the predominant mind-share today.

To many enterprises, LDAP is their one identity hammer. And they see all their identity problems as nails. If we want them to put down the LDAP hammer and pick up the IGF pneumatic impact wrench, we have to explain to him in real world business cases why it’s better. Because they  know the LDAP hammer will work and they already have it in their tool box. The IGF pneumatic impact wrench is a strange new tool to him that they must first understand and second justify purchasing.

Of course AD isn’t all identity in an enterprise. But for extranet identities you will have to justify why IGF or some other identity service is better than just throwing in an ADAM, OID,  or OpenLDAP instance. Or even a virtual directory like OVD or OptimalIdM VIS (for the .NET centric customer). The enterprise architects know they already have a wide variety of tools and APIs to leverage LDAP. They don’t yet have those for IGF and other identity services.

Bottom line – identity services will not reach the level of adoption to where you could say there is an “Identity Bus” until there are compelling business cases made for it. Enterprises not only have to adopt the identity service standards, but they need to make vendor support of those standards a selection criterion to drive adoption.