In the previous post, we discussed quality assurance, and all the steps involved in making sure the report is functioning to the businesses expectations. If you completed phase 3 with rigor, this next phase should be a cinch! If you haven’t, please make sure to go back and read the previous 3 posts in the series.
- Phase 1: https://datastrides.wordpress.com/2019/02/02/your-report-2-0-a-data-engineers-guide-to-migrating-bi-reports-from-old-to-new-systems-part-1/
- Phase 2:https://datastrides.wordpress.com/2019/02/23/your-report-2-0-a-data-engineers-guide-to-migrating-bi-reports-from-old-to-new-systems-part-2-building-the-extract/
- Phase 3: https://datastrides.wordpress.com/2020/03/26/your-report-2-0-a-data-engineers-guide-to-migrating-bi-reports-from-old-to-new-systems-part-3-qa/
Congrats! The report is live, and you have succeeded migrating it! (Of course with the help of your dev-ops engineers, ensuring you have a seamless CI/CD process in place – this is for another blog post). But before we close, there is one final phase that we cannot neglect: Supporting and enhancing the report.
No matter how diligent you are in writing good code, there are always issues that will arise and additional enhancements business users will want. Thus, it is always a good idea to have a solid process in place for when you inevitably have to fix a bug or have an enhancement to make.
First we will discuss supporting the report, or in other words, bug resolution.
One scenario in this category is ‘the report is down’. These are scary words, but make sure to stay calm, and go through the simple troubleshooting steps.
Find out exactly what this issue is. In the eyes of a business user, ‘the report is down’ could mean anything from the report being unreachable via normal channels, to the data is not current. Remember there are a lot of steps that go into generating the report. Some things to check:
- Can you reach the report?
- Is the reporting server down?
- Is the report deployed correctly?
- Is the extract failing?
- Is the ingestion of the extract failing?
- Is the ETL that populates the DataMart or Data Warehouse failing? This could account for missing data.
Once you can narrow down the issue you will have an easier time resolving it. There are always to steps to take to resolve any problem. Let’s say the ETL was failing because a table was not deployed properly in your previous release, thus the data was not current. Step 1: Hotfix the issue by manually deploying the table, and re-triggering the load. Step 2: Determine the root cause of the issue. WHY was it not deployed properly? Until you fix the root issue, the problem could persist. This may seem obvious, but sometime finding root issues can be difficult, so it is easy to chalk it up to a fluke.
The above situation is an ad-hoc / hotfix scenario. However, the goal should be to catch errors or issues before the end users do. In all of my ETL solutions, I like to have logging / error handling in as many places as possible.
- Source systems are down
- ETL into the Data Warehouse
- ETL into the Data Mart
- Extract into the report
- Report itself is down
If any of these steps fail, I would like emails automatically sent to me / the on call team. This way, once you get an email like this, you can send a note out to the business users immediately notifying them of the issue. This pro-active approach will save you a lot of recourse from the business and will protect all of your downstream users from wasting time figuring out why things look strange.
The best thing you can do when there are issues with a report is communicate appropriately. For the most part, people are understanding that things can go wrong. But if you pretend things are ok, and people waste there time looking at bad data, the sentiment can turn negative quickly.
The other type of common bugs are data related issues. These can happen anywhere in our pipeline, and though it can be tedious, your best bet is to trace the data backwards until your find where the issue occurred.
Once the report is stable, the business users will begin to want additional enhancements to the report. Whether this is as simple as adding an additional field, you should NEVER make changes directly in production, unless you are absolutely required to. As with all good software development processes, the goal will be to have some sort of source control (git) flow, that looks like the following:
- Dev – the environment where you make the changes and test them out.
- Test – the environment where QA and the end users test and verify the changes.
- Stage – production level data in a non-prod environment.
- Prod – Once code is thoroughly QA’d and approved it can go to prod.
One challenge you might face here is that lower environments might not have the quality / quantity of data that production has. This will directly impact the way your reports look in lower environments. This is especially prevalent in scenarios where the Warehouse or Data Mart is being developed at the same time as the report. In this scenario, where data is paramount, I recommend having a fourth environment called Stage which houses all of the latest production data and is a final staging area for end users to test reports before they go to prod. This assures that when the report goes to production, what the users saw in the stage environment will reflect exactly the same in production, because the underlying data is the same.
When discussing enhancements with the business, always have clear requirements and clear priorities, and as I have mentioned in the past, document everything! Remember the detailed design we created earlier? Any enhancements should be documented thoroughly there before any coding begins. Basically, to add enhancements, go through the first 3 phases again, but stay especially focused on the new changes.
Hopefully, this will ensure smooth sailing as you continue to add new features
As with any technical solution, there are many ways to achieve the same result. I hope this series conveyed not exactly how things have to be done, but rather provided a framework to approaching BI development.
Please feel free to comment if you have any additional thoughts or questions!