Unlocking Back Office Data through Salesforce Lightning Connect

The general approach to enterprise programming is changing. Significant investments are being made in making enterprise IT applications agile and nimble. Salesforce, being the innovators they are, have taken significant strides in this direction. One such move is the introduction of the lightning framework – a philosophy pioneered to reduce the time in developing and deploying enterprise applications.

For any enterprise system one of the key success factors is its ability to integrate with other systems; and how easily it can do so. Ironically, integration to external systems is usually the primary bottle neck in most enterprise IT projects. Salesforce has reimagined enterprise integration through Lightning Connect. Lighting Connect questions the fundamental tenet of data integration – Do you really need to integrate data, when the only need is to be able to view the data? Would it not suffice if you are able to see the external data within your system, in the way you want to?

If you think about it, you would realize this is just about enough for many integration use cases. Often you would be faced with the need of showing data from a back office ERP system in Salesforce. If the data only needs to be visible, that is not the same as bringing the data from the external system into the Salesforce database.

Lightning Connect gives you the capability to easily consume external data in the form of an OData service, and render this data as Salesforce objects – with point-and-click ease. With lightning connect, it takes about 5 minutes and a few dozen mouse clicks to integrate an external data system – provided it can be read via an Odata service.

The OData Paradigm

Conversations about Lightning Connect are never complete without a mention about OData. OData – or the Open Data Protocol – is a relatively new application data interfacing model pioneered by Microsoft. By definition it is an open protocol to allow the creation and consumption of query-able and interoperable RESTful APIs in a simple and standard way. Once an application or a data system is enabled for OData, each of its data elements can be accessed via an HTTP URL. For example, the following is how a typical OData URL would look like:

http://services.odata.org/Northwind/Northwind.svc

This is the single URL for accessing a fictitious DB called Northwind. With proper parameters, you would be able to access any data within this database, in the form of a URL. For instance:

http://services.odata.org/Northwind/Northwind.svc/Customers(‘ALFKI’)/$links/Orders

would give you all Order records related to the ALFKI customer.

While Salesforce does a good job of marketing Lightning Connect as a mechanism to create integration in a blitz, an implicit fact that comes along is that – to be able to do this, the data from your source application must be exposed as an OData service, similar to the Northwind OData service shown above.

Some enterprise suites already expose their data through OData services – like SAP Netweaver Gateway, Microsoft SQL Server, Dynamics CRM, Azure DB, IBM Webshphere extreme Scale and Heroku Connect External Objects. While for others, you need to develop & enable an OData service. There are multiple ways of setting up an OData service to expose application data. You could either choose to build it yourself using the multiple developer frameworks available. The prevalent ones are .Net WCF Data Services, Java (odata4j), and NodeJS. Another is the .Net WCF Data Services package, which gives ready-to-use templates that makes authoring OData ready code a breeze. Microsoft Visual Studio 2013 Community Edition ships with this package pre-installed, in case you want to give it a try. If your organization doesn’t have the skill set to build an OData service from scratch, you could always rely on the multiple partners that provide this service – like Dell Boomi, Informatica, Jitterbit, and MuleSoft.

Okay I have an OData Service. Now What?

Assuming you have an OData ready data source, let us discover the real power of Salesforce Lightning Connect. Imagine there is a fictitious database, which contains Financials transactions of your customers – and is readable through an OData service. Assume there is an Invoice table within this databaes, which contains all invoicing information related to the customers. The use case is to make this invoice data visible within Salesforce, using Lightning connect.

The first thing you would need to do is to notify Salesforce about your external data source. You would do this by navigating to Setup | Develop | External Data Sources. Create a new external data source and provide your base OData URL.

You would then click the “Validate and Sync” button, which would take you to a meta-data list showing the available tables within that database. Select the ones that you want to show within Salesforce, and click the “Sync” button. Remember to also choose any intersection tables that are required.

Once done, you would notice that Salesforce has intuitively created the External Objects for your use based on your table structure. The columns transform to Salesforce fields, and tables transform to external objects – thereby bringing your external system within the MVC paradigm of Salesforce.

Shown below is the view of the external object that was just created.

Like you would notice, an external data object looks just like a regular Salesforce object, except that it is suffixed with “__x” instead of __c. This object can be used just like any other Salesforce custom object – to create custom fields, custom tabs, make look up associations, reference in Apex or VisualForce, and so on. Thus, there is almost no additional learning curve to use the external data, other than the regular knowledge of Salesforce customization.

But remember, Salesforce only creates the external objects; and not the associated object relationships. So you would need to re-create any relationship that existed between your external tables, by creating “External Lookup” fields on the external objects.

Similarly, you could create an “Indirect Lookup Relationship” field, on your external object to link it to any Salesforce objects. This means, for our use case in context, you would be able to show the invoice details specific to your contact, as a child applet on the contact detail page.

Of course, to create an indirect look up relationship between a Salesforce object and an external object, there needs to be an external ID (like a foreign key), that can be used to match the data. In our case, let us assume the Salesforce Contact ID is available as a column in our Invoices table. This shows a simple use case wherein, you are able to display your back-office data within Salesforce, without having to author any integration code.

More Lightning Connect Mojo

The key point to understand about a lightning connect integration is that – the data is not copied into Salesforce. The data is queried and rendered real time, on each page load. Besides helping you save valuable data space that you have bought in the form of Salesforce licenses, this also lets you bypass the regular API governor limits. So for the very same page that you see in the figure above, if you were to do a page refresh, the Invoice data is actually queried & rendered again. Thereby, you are always looking at live and up to date external data – without having to author separate batch syncing programs to keep the data up to date.

An interesting aspect about Lightning Connect is that, the external objects are readily available for Salesforce1. Which means, Lightning Connect is also an effective mechanism to make your legacy back-office data mobile ready.

Lightning Connect is also an answer to data residency concerns that could occur in a multi-geography multi-company environment. There could be governance that mandate data should not be copied outside of a particular system, or particular geography. Since Lightning Connect does not copy any data between systems, you are likely to stay aligned to any data residency norms that may apply.

As much as knowing how to use an external data object, it is also of value to understand where you can use them. As of today, it is possible to use the external objects to create:

• Tabs

• List Views

• Detail Pages

• Chatter Feeds

• Visualforce pages

• Apex SObject types

• REST/SOAP API access

• SOQL and SOSL queries from Apex or API

• In Salesforce1 Mobile Platform

Limitations of Lightning Connect

Being a recently launched feature, there are some limitations with the current capabilities of Lightning Connect. To start with, it works based on OData v2, while the current standard has moved to v4. This means that, it is not possible to perform write operations onto the external data using Lightning Connect. The present framework only supports a read-only display of external data in Salesforce.

Another limitation is the fact that external data objects cannot be used in reporting as of now. Neither can it be part of Formula, Roll Up Summary Fields, Triggers, Workflows, Approval Processes, Validation Rules or Field History tracking.

Most of these are already on Salesforce roadmap, with the capability to Edit External Data, and Report on External Data topping the priority list. Once these gaps are bridged, Lightning Connect would definitely have the potential to redefine enterprise integrations.

Salesforce1 Lightning Connect is available for an extra cost in Enterprise, Performance, and Unlimited Editions; and is priced per data source. It is available for free in the Developer Edition, with the limitation of being able to add just 1 data source.