Our Experience Integrating HR Tech for People Analytics - a Vendor by Vendor Comparison.
This will be a living post and we will continue to update as we have time to collect thoughts on each vendor and as we complete integrations to new vendors. Not every source we work with will be listed here but we'll cover the majors ones that we often work with.
At One Model we get to see the data and structure from a load of HR systems, and beyond, basically anything that holds employee or person data is fair game as a core system to integrate for workforce analytics.
After more than a decade of experience directly integrating data from these systems into analytics and reporting solutions, we have a lot of experience to share.
Below I'll share our experience with highlights from each system and how they align with creating a people analytics warehouse. Some are better than others from a data perspective and there's certainly some vendors that are yet to understand that access to data is already a core requirement of buyers looking at any new technology.
Bookmark this blog, add your email to the subscription email list to the right, or follow me (Chris Butler) on LinkedIn to stay up to date.
[A note on ratings - Ratings are provided as an anecdotal and unscientific evaluation of our experience in gaining access to, maintaining, and working with the data held in the associated systems. They are my opinions.]
If you would like to make use of any of our integrations in a stand alone capacity, we now offer a data warehouse only product where you utilize just our data pipeline and modelling engine to extract and transform data into a data warehouse hosted by One Model or your own data warehouse. We'll be releasing some more public details soon but you are a company that likes to roll your own analytics, visualizations, and just need some help with the data side of the house, we can certainly help. Contact Us
- One Model rating - 2.5/5
- Method - API for standard objects, built in reporting for custom objects (via reporting-as-a-service, or "RaaS")
- The Good - Great documentation, Easy to enable API access and control accessible fields, Good data structures once you have access. The RaaS option does a good job but is limited.
- The Bad - Slow; Slow; Slow; No custom fields available in API, Geared towards providing a snapshot, number of parallel connections limited, constant tweaking required as new behaviors identified, Expert integration skills required; True incremental feeds require you to read and interpret a transaction log
Workday analytics embedded into the product is underwhelming and we're yet to see Prism Analytics make a dent in filling the needs that people analytics teams or HR analysts have beyond convenience analytics. So in the meantime if you are serious about improving reporting and people analytics for Workday you're going to need to get the data out of there and into somewhere else.
On the surface Workday looks to have a great API, and the documentation available is excellent. However the single biggest downfall is that the API is focused on providing a snapshot, which is fine for a simple list reports but does not allow a people analytics team to deliver any worthwhile historical analysis. You don't get the bulk history output of other systems or the ability to cobble it together from complete effective-dated transactions across objects. To capture the complete history we had to build an intense process of programmatically retrieving data, evaluating, and running other API calls to build the full history that we need. If you want more detail take a look at my blog post on the subject The end of the snapshot workday edition.
The complexity of the integration therefore is multiplied and the time taken suffers immensely due to the object oriented architecture that requires you to load each object into memory in order to be able to retrieve it. A full destructive data extraction means you're looking at 8+ hours for a small-medium enterprise, and expanding to a week if you're a giant. The problem is exacerbated by the number of parallel connections allowed running at a fraction of the stated limit. A full historical API integration here is not for the faint of heart or skill, we have spent 12+ months enhancing and tweaking our integration with each release (weekly) to improve performance and solve data challenges. Our integration to give a sense of scale generates some 500+ tables that we bring together in our modelling engine in preparation for analytics.
Out of the box integration plugins are going to be focused on the snapshot version of data as well so if you don't have the integration resources available i wouldn't attempt an API integration. My advice is to stick with the built in reporting tools to get off the ground. The RaaS tools do a good job of combining objects and running in a performant manner (better than the API). However they will also be snapshot focused and as painful as it will be to build and run each timepoint you will at least be able to obtain a basic feed to build upon. You wont have the full change history for deeper analysis until you can create a larger integration, or can drop in One Model. Robert Goodman wrote a good blog a little while back looking at both the API and his decision to use RaaS at the time, take a read here. Workday API vs RaaS
Regardless of the problems we see with the architecture the API is decent and one of our favorite integrations to work with. It is, however, little wonder that with the data challenges we have seen and experienced, that half of our customers are now Workday customers.
Figure 1: One Model's data extraction from Workday
- One Model rating - 4.5/5
- Method - API
- The Good - A dynamic API that includes all custom MDF data!! Runs relatively quickly; Comprehensive module coverage;
- The Bad - Several API endpoints that need to be combined to complete the data view; Can drop data without indication; At times confusing data structures
4.5 out of 5 is a pretty phenomenal rating in my book. I almost gave SuccessFactors a perfect 5 but there's still some missing pieces from the API libraries and we've experienced some dropped data at times that have required some adaptations in our integration. Overall, the collection of SF API's is a thing of beauty for one specific reason: it is dynamic and can accommodate any of the Meta Data Framework (mdf) custom changes in its stride. This makes life incredibly easy when working across multiple different customers and means we can run a single integration against any customer and accurately retrieve all customizations without even thinking about them. Compared to Workday where the API is static in definition and only covers the standard objects this facet alone is just awesome.
This dynamic nature though isn't without it's complexities. It does mean you need to build an integration that can interrogate the API and iterate through each of it's customizations. However, once it is complete it functions well and can adapt to changing configurations as a result.
Multiple API endpoints also require different integrations to be merged. Tthis is a result of both upgrades in the API's available in the case of the older SuccessFactors API and the OData API as well as providing an API to acquired parts of the platform (i.e. Learning from the Plateau acquisition). We're actually just happy there is now an API to retrieve learning data as this used to be a huge bug bear when I worked at SuccessFactors on the Workforce Analytics product. The only SF product I know of right now that doesn't have an ability to extract from an API is Recruiting Marketing (RMK) from the jobs2web acquisition, hopefully this changes in the future.
Full disclosure, I used to hate working with SuccessFactors data when we had to deal with flat files and RDF's, but with the API integration in place we can be up and running with a new SuccessFactors customer in a few hours and be confident all customizations are present.
Another option - Integration Center
I haven't spoken here about the new Integration Center release from earlier last year as we haven't used it ourselves and only have anecdotal evidence from what we've read. It looks like you could get what you need using the Integration Center and deliver the output to your warehouse. You will obviously need to build each of the outputs for the integration which may take a lot of time but the data structure from what i can tell looks solid for staging into an analytics framework. There is likely a lot of tables to extract and maintain though, we currently run around 400+ tables for a successfactors customer and model these into an analytics ready model. If anyone has used the Integration Center in an analytics deployment please feel free to comment below or reach out and i would be happy to host your perspective here.
Figure 2: One Model's data extraction from SuccessFactors
Oracle HCM Cloud (Fusion)
- One Model rating - 2/5
- Method - HCM Extracts functionality all other methods discounted from use
- The Good - HCM Extracts is reasonable once you have it set up. History and all fields available. Public documentation.
- The Bad - User interface is incredibly slow and frustrating. Documentation has huge gaps from one stage to the next where experience is assumed. API is not functional from a people analytics perspective: missing fields, missing history, suitable only for point-to-point integrations. Reporting/BI Publisher if you can get it working is a maintenance burden for enhancements. HCM Extracts works well but the output is best delivered as an XML file.
I think I lost a lot of hair and put on ten pounds (or was it ten kilos?!) working through a suitable extraction method for the HCM Cloud suite that was going to give us the right level of data granularity for proper historically accurate people analytics data. We tried every method of data extraction from the API to using BI Publisher reports and templates. I can see why people who are experienced in the Oracle domain stick with it for decades, experience here is hard won and akin to a level of magic. The barriers to entry for new players are just so high that even I as a software engineer, data expert, and with a career spent in HR data many times over, could not figure out how to get a piece of functionality working that in other systems would take a handful of clicks.
In looking to build an extraction for people analytics you have a number of methods at your disposal. There's now an API and the built-in reporting could be a reasonable option for you if you have some experience with BI Publisher. There are also the HCM Extracts built for bulk extraction purposes. We quickly discounted the API as not yet being up to scratch for people analytics purposes since it lacks access to subject areas, fields, and cannot provide the level of history and granularity that we need. I hope that the API can be improved in the future as it is generally our favorite method for extraction. We then spent days and probably weeks trying to get the built in reporting and BI Publisher templates to work correctly and deliver us the data we're used to from our time using Oracles on-premise solutions (quite a good data structure). Alas, this was one of the most frustrating experiences of my life, it really says something when I had to go find a copy of MS Word 2006 in order to use a plugin that for some reason just wouldn't load in MS Word 2016, all to edit and build a template file to be uploaded, creating multiple manual touchpoints whenever a change is required. Why is life so difficult??
Even with a bunch of time lost to this endeavour our experience was that we could probably get all the data we needed using the reporting/BI publisher route but that it was going to be a maintenance nightmare if an extract had to change requiring an Oracle developer to make sure everything ran correctly. If you have the experienced resources this may work for you still.
We eventually settled on the HCM Extracts solution provided that while mind-numbingly frustrating to use the interface to build and extract will at least reliably provide access to the full data set and deliver it in an output that with some tooling can be ingested quite well. There's a number of options for how you can export the data and we would usually prefer a csv style extraction but the hierarchical nature of the extraction process here means that XML becomes the preferred method unless you want to burn the best years of your life creating individual outputs for each object tediously by hand in a semi-responsive interface. We therefore figured it would be easier, and enhance maintainability if we built our own .xml parser for our data pipeline to ingest the data set. There are .xml to .csv parsers available (some for free) if you need to find one but my experience with them is they struggle with some files to deliver a clean output for ingestion.
With an extract defined though there's a good number of options on how to deliver and schedule the output and reliability is good. We've only had a few issues since the upfront hard work was completed. Changing an extract as well is relatively straightforward if you want to add a field or object you can do so through the front end interface in a single touchpoint.
We do love Oracle data don't get me wrong - the construction and integrity is good and we have a repeatable solution for our customer base that we can deliver at will, but it was a harrowing trip of discovery that to me, explains why we see so few organizations from the Oracle ecosystem that are out there talking about their achievements. Don't make me go back, mommy!
ADP Workforce Now
Applicant Tracking Systems