So my head is spinning more than a little after this week. I’ve had a data strategy meeting with whom and for what purpose I genuinely do not know. Then three or four separate data model development meetings then reviewed my teams work… All of this has made me feel it is time to share my design philosophy and how I go about building Power BI reports.
I use a four step process and for my personal opinion that fits the product best, the key thing to remember is that you do not always have to go forward, I’m a huge believer in emergence and emergent behaviours, BI is a prime emergence environment
Emergent behavior is behavior of a system that does not depend on its individual parts, but on their relationships to one another. Thus emergent behavior cannot be predicted by examination of a system’s individual parts.
In other words as reports develop and value is shown addition relationships or correlations become clear. This understanding is what drives my design principles, hopefully even if you disagree you will appreciate some of the ideas and can incorporate them into your own modelling principles.
By dividing the report creation process into four distinct stages expectations and delivery gateways established, typically you should try to sign off each stage before moving forward, but you have to accept the reality that emergence willl often move you backwards, that is fine and must be accepted, pushing on regardless to the recipie for disaster.
This is quite common issues as you start to go forward in Power BI, so I’ll try and scope it out for you guys.
A client has five or six data sources coming together to produce a Power BI report, due to the nature of the data sources they return a “now” state so trending is a challenge, typically to set trending up we would set up a snapshot. In this data model data is coming in from a range of services not all of which are in SQL some data is coming directly from products as well. The net result is that snapshotting would be complex and ultimately remove the benfit of using Power BI and the Mashup engine. Instead Patrick spelled it all out in a Guy in a Cube Video. Now we didn’t need to do all of this, but something went “ping” and suddenly I had a vision of the future. We could use the integrated SharePoint with the App space to host a SharePoint List and then use the SharePoint List to bring the data back.
That’s dumb, why would I want to do it all this way, SharePoint isn’t going to hold the data I need!
The biggest benefit of using this method is that quickly and easily trends can be set up based on complex “unrelated” data.
Watch this space as I’ll keep adding to it as we find things out about using the mechanism after all there isn’t really a short cut to setting up a trend and getting real world data.
SO reality hit yesterday, we needed to import the data from all a clients Active Directories – not many by any standards and all 2008 I still have nightmares about uncovering forgotten NT4 domains that are “business critical” – anyway you can use Power BI Desktop to easily import multiple Active Directories and then merge them into a table for reporting, that all works really well and is frankly a breeze. You say you want to use an Active Directory, give it some credentials and then select where and what you want to import. If you need to combine data then you just merge your queries.
So why are you wasting our time by making us read this?
It all falls apart when you start to set it up into an App ready to publish it. The Enterprise Gateway (On-Premise Gateway) doesn’t support active directory connections, so you have two options
Hire someone to manually refresh reports all day – This is a great fun job and I’m sure your staff retention will be sky high
Use a single Windows account for everything (that needs Windows accounts) including logging into Power BI and all your data sources – I can hear the panic in anyone with a security background reading this
Well neither of those options work for my company! So we chose not to choose naff, we chose life instead
Nope, if you came here looking for a fan site, I’m afraid this isn’t what we are, CSI or Continual Service Improvement is a process of structured change and development that ultimately leads to the improvement of a service being delivered.
The human brain sees “improvements” automatically, I’m sure we’ve all received a set of directions and having completed the journey a few times thought “I’ll bet if I just change… I would get to my destination quicker/safer/more economically”. That’s the simplest form of “Improvement” I can think of. The more analytical of you may have timed your journey different routes or used another benchmark (was it by a busy road, how much petrol did I use) the point would be you are actually making a comparison, the net result is that you are able to look at you journey times and make and informed decision on which route is the most appropriate. A few weeks/months later no doubt some more changes come to mind or open up or there are road works on your “optimal” route, does that mean having changed once that you can’t change again? Heck no, you’ll change your route to the best, who knows you many have even found that a different route was always preferable at key times/days. What you’re actually doing is living the Continual Service Improvement (CSI) process.
As you progress in Power BI (and Power Pivot), you quickly find you need a date dimension, the problem is that date dimensions are difficult to build and even more difficult if you want them to be more than just a list of dates.
“Stop I’m confused… how can a list of dates be more?”
The role of a good date dimension is indeed to be more than just a list of dates, there are functions in Power BI to produce a list of dates, the issue is that you want something that is more. I’ll be honest at this point before we dive into the queries, I use, I cannot remember where I got the original source, I found three or four different date dimension SQL queries initially and then started pulling them together and adjusting them to fit my needs. For me we were going to have the majority of our data coming from a SQL server so it makes sense to have a SQL date dimension, based on what we did and after doing some benchmarking within Power BI I would recommend housing it on a Database, putting it in Power BI or excel won’t stop things working, but I have got best results from this method.
The important thing about the first bit is that there are many, many scripts out there to do what you need. Typically they have the same format.
I’ll make these source files available to download
Populate Dates Dimension
Deal with 1 character days and months
Set Fiscal Year Months – where I work we use August to July so I needed to add a case statement here, case statements sound more complex than they are