Category Archives: OASIS

SPML and DSML search filters not so hard

One issue that has been raised in regards to SPML is search filters. SPML allows searches that optionally specify a starting point (in terms of an SPML container), a subset of data to return, and a search filter. In the DSML Profile, the search filter is naturally a DSML filter.

DSML filters can be arbitrarily complex, just like the LDAP filters they model. For instances a DSML filter could be something like “get everyone with the last name of smith”, or in DSML:

<filter xmlns=’urn:oasis:names:tc:DSML:2:0:core’>

<substrings name=’Name’><final>Smith</final></substrings>


Or it could be “get everyone with last name smith not located in Orlando”:

<filter xmlns=’urn:oasis:names:tc:DSML:2:0:core’>


<substrings name=’Name’><final>Smith</final></substrings>


<substrings name=’Office’>






Now if your back end data is stored in LDAP, then this is pretty easy to handle. Just convert to an LDAP filter and do any attribute name mappings required. If you backend data is SQL, it just slightly more difficult to translate the DSML filter into a SQL query clause.

But what if your back end data store doesn’t support a query mechanism? What if the data is in a flat file, or a NOSQL DB? What if the data is only accessible through an API that doesn’t allow for filtering?

There are several ways to solve that problem, but the easiest is to recursively walk the DSML filter and create a decision tree where each node determines if a given instance passes the part of the filter it knows.  The code for this is pretty simple in .NET and I posted an example here. Note that this example is just a partial implementation of the SPML search request for the purposes of demonstrating this concept. It is not a full featured implementation of SPML.

The basic idea is that an abstract data provider would return a dictionary of the attribute values for each entry in the data. The interface could look like (in C#):

public interface IUnfilteredDataProvider {

List <DSMLData> DoUnfilteredSearch();


In this example the sample data provider reads entries from a flat file. On each search request the filter is recursively read and turned into nodes in a decision tree. Each data entry is then passed to the decision tree and if it passes the filter it is appended to the returned results:

List<DSMLData> dataList = this._unfilteredProvider.DoUnfilteredSearch();

DataFilter df = GetDataFilter(searchRequest);

List<PSOType> returnPSOs = new List<PSOType>();

// return only those entries that pass the filter

foreach (DSMLData data in dataList) {

if (df.Pass(data)) {




The GetDataFilter method walks the DSML filter and constructs a decision tree (feel free to download the sample and look at the code for more details). No special meaning given to any of the attributes returned by the provider. They are all just treated as DSML attributes. Of course you will note a potential scalability issue with large data sets, but there are several tricks that can be used to minimize that (thoughts for a later post).

Oh, and this approach works great for creating a DSML service as well and the general concept would be just as easy to implement in Java.

So what does all this mean? Supporting filtered searches in an SPML or DSML service is really not that hard, even if your data is stored in a data store that does not support filtering.

SPML SIG at Catalyst

Bob Blakley announces here that the Burton Group identity blog has transitioned to individual Gartner blogs. He also announces an SPML SIG at the next two Catalysts.

It’s good to see attention being given to SPML again. But will this translate into real movement to adopt SPML (either 2.0 or a to be developed 3.0)? Perhaps, but we may have to take a step backwards in order to move forwards. If work starts on SPML 3.0 then that will effectively kill adoption of 2.0. But if 2.0 isn’t being adopted anyway, why not go ahead and do a 3.0?

Interesting times.

SPML 3.0 in 3D!!!

OK I kid about the 3D, but I am starting to hear from various identity folks that it’s time to start thinking about SPML 3.0. The latest is John Fontana’s post on that here.

While I don’t think that there are any technical reasons SPML 2.0 can’t be used for interoperable provisioning, the market has clearly not embraced it yet. There are some SPML enabled products out there, but not nearly enough to reach the critical mass that is needed.

So would an SPML 3.0 effort succeed where SPML 2.0 has so far not succeeded? I honestly can’t say, but I feel it’s worth giving it a go. The industry really needs this. My employers products need it.

DSML Revisited

There was an interesting article on DSML recently (I ran across it in Dave Kearn’s IdM Journal). This is a useful summary, but unfortunately the article mixed in references to DSML v1 and DSML v2 in a way that will likely be confusing.

Just to clear this up, DSML v1 was a private vendor spec originally initiated by DSML v2 is an OASIS Standard. There are some key differences DSML v1 only supported representing LDAP data and schema in XML but did not address web services. DSML v2 addressed web services. The two specifications are not compatible. The free DSML tools mentioned in the article are actually DSML v1 tools.

DSML v2 is supported by some of the main LDAP Vendors. DSML interfaces to LDAP servers include the Microsoft DSML Services for Windows (DSFW), the Netscape DSML Gateway, and the Novell DSML Support. Microsoft has also added DSML support to its .NET 2.0 Directory Services, although I haven’t had a chance to play around with it.

I had some peripheral involvement in the DSML v2 effort. While at Access360 I designed the new protocol for the enRole IdM server to communicate with the enRole provisioning agents. I create an XML/HTTPS protocol that was called the Directory Access Markup Language (DAML). DAML was basically a web services layer on top of DSML v1. It was actually pretty cutting edge at the time; the enRole system was using an XML/HTTPS protocol for provisioning while SOAP was still in development as standard. Access360 was of course acquired by IBM and the enRole system became the Tivoli Identity Manager (TIM). I don’t know if the DAML protocol is still being used, but you are likely to still see references to it in TIM documentation.

When the DSML v2 effort started with OASIS, the Access360 DAML spec was contributed as input to DSML v2, along with the Dir-XML spec from Novell. I had some initial involvement with the DSML v2 effort, but that ended when I left Access360 and joined OpenNetwork Technologies.

But I have always been a fan of DSML, which is why (for better or worse) SPML 1.0 was based on DSML v2.

(Mirrored from TalkBMC)

, , , , , ,