DHW - DataHandwerk toolkit mssql
Key contents of the architecture
|
Read the Architecture documentation to learn why DataHandwerk exists, how it works and is built. Some key content from the architecture documentation: |
DHW (DataHandwerk) is designed to support BI and DWH developers working with the Microsoft BI platform (MS SQL Server).
Handwerk statt Massenproduktion. - Craft instead of mass production.
MSSQL based tools for data craftsmen. - Individual customization instead of assembly line.
|
Handwerk = craft Craft is the term used to describe numerous commercial activities that manufacture products mostly to order or provide services on demand. Craft activities are contrasted with industrial mass production. Als Handwerk werden zahlreiche gewerbliche Tätigkeiten bezeichnet, die Produkte meist auf Bestellung fertigen oder Dienstleistungen auf Nachfrage erbringen. Die handwerkliche Tätigkeit steht der industriellen Massenproduktion gegenüber. |
The project’s key stakeholders are DWH developers who see their work as a craft.
-
A toolkit is created for these data craftsmen and gradually filled and expanded with tools.
-
The data craftsmen are given the possibility to work faster and better with the toolkit and to automate parts if (and only if) they wish to do so.
-
Tools and methods can be used individually and independently optional. They do not have to be used in a mandatory way.
-
The toolbox should always be perceived as an enrichment and aid, gladly as a supportive, never as a constricting corset.
A developer is not undesirably restricted by the use of tools and methods. He allows himself to be partially and consciously restricted if the advantages subjectively outweigh the disadvantages for him.
Alternatives
Data Warehouse Automation, repository based database generators
There are commercial tools that define the structure and function of relational and analytical databases via a repository in order to generate the code of these databases.
| These tools are one-way: repository ⇒ target database. |
One can work very successfully with these tools if one accepts the methods and approaches provided.
You can compare these tools with rail networks. As long as you are satisfied with these networks and the places connected to them, you can travel more or less effectively and efficiently with one or the other railroad in one or the other rail network. Problems arise when places or routes are missing or the connection between places becomes more cumbersome than one would like. Therefore, roads, trucks and cars also have great justification and importance. Sometimes there are even only footpaths.
My subjective praise of AnalyticsCreator can be found here (in German): Warum ich seit 2017 den AnalyticsCreator verwende.
-
Deltamaster Modeler (Additional program for the Delta Master)
Database documentation
The database documentation was created with the Database documentation generator: docs.usp_AntoraExport - P
The documentation also contains
-
real and virtual PK (only for tables and views)
-
column references extracted and used with SqlParse (only for tables and views)
-
Visualizations of FK, object and column references with PlantUML
-
formalized Procedure steps for procedures, created with the Procedure Generator
Development status
primary project location: https://gitlab.com/datahandwerk/DataHandwerk-toolkit-mssql
You can read why the project exists and how it developed in the following blog posts (in German):
| The project was launched at the beginning of 2021 and is in the alpha stage. The main focus is currently the implementation. Various refactorings are still taking place in the database. The documentation is being created and is still very incomplete. |
Project communication
The preferred channel for project communication is https://datahandwerk.zulipchat.com/