Many blog posts and tutorials on Qlik Sense focus on the data visualisation features. The load script is often omitted maybe because the script is almost identical to the QlikView load script, of which much has already been written. This post highlights some of the techniques that are common to both products.
The Importance Of Solid Script
Qlik Sense has a very intuitive and easy to use UI for building great apps. You can only build great apps if you have the right data available. This generally comes down to the correct work being done in the load script. With QlikView I advised that you should spend at least half of your development time in the load script.
That ratio hasn’t really changed with Qlik Sense (though the ability to segregate the work between separate developers and designers is even more encouraged through the structure of the software).
This is why many of my previous blog posts have focused on script and preparation. This blog post links together some of the best of these for Sense developers.
The use of a well structured data tier is good practice in much software development. Sense is no different. Building a QVD layer to persist data and share them between apps is almost essential. The hub UI and app structure does not encourage it (how often do I need to be reminded that my ETL app contains no data) but you should do it all the same.
Once you have the QVD layer you can build an incremental load. This post describes how. Just be aware of the need to use the “lib” functionality in Sense when you store and retrieve your QVDs.
Loading Many Files From A Drop Folder
If you have many files that arrive, perhaps daily, into a folder that you need to parse; Sense can enumerate around them and load each in turn in the same way as QlikView. You can build a structure that only picks up and processes new files, persisting previous data back into QVDs.
This post describes how, just watch for the lib reference when referencing files – regardless of where in the code it is referenced.
The Use Of ApplyMap
After many years of Qlik development I still think the implementation of ApplyMap in the Qlik Platform is superb. You can merge data from multiple sources with more flexibility and without the risks associated with a SQL type JOIN.
This post gives a lot of detail about the syntax of ApplyMap, and some neat tricks you can do with it.
Further Use Of The STORE Statement
The posts above all reference the STORE statement for persisting data to QVD. There is much more that you can do with this statement though, such as exports to be used in other products by writing to CSV or tab delimited files. The post also has a neat trick in it for creating dynamic web pages (with data in them) from QlikView.
The same techniques are all possible in Qlik Sense also.
Another neat feature of the Qlik load script is the ability to do preceding loads. This is where the output from one part of the load is used in another load, making it very easy to cut down on code re-use and produce more readable source code.
Preceding loads can be used in Qlik Sense in exactly the same way as QlikView.
Perfecting Your Data Model
Much is written (especially on this blog!) about getting your data model correct in QlikView. The reason for this is that it can be the difference between an app that works and one that doesn’t.
QlikView and Qlik Sense share the same back end, so the same considerations around getting your data model correct are needed. If you are a Sense developer you could do much worse that reading up on QlikView data modelling. This post is just one of the many you may choose to read.
by Steve Dark - Jul 4, 2016