In the digital age, the impact of data is becoming increasingly profound. By the end of 2022, a myriad of big data artificial intelligence models had emerged with varying degrees of quality. The effectiveness of these models highly depends on the quality of the input data, thereby imposing higher standards for data quality. However, people’s understanding of the essence of data is still at a relatively primitive stage. This has led to three major issues in existing data processing techniques: The first issue is the ignorance of the inherent uncertainty present in data; the second is a lack of effective methods for predictions; and the third is overlooking the bias introduced by the human-data relationship.
To address these issues, on October 9, 2023, Mongoose Think Tank officially launched TDRU (Tools of Data Reconstruction on Uncertainty) – a tool based on the principle of uncertainty that aims to solve the aforementioned problems through data reconstruction.
Theoretical Background of TDRU
Over time, Mongoose Think Tank has researched the sources of uncertainty extensively. In the field of data science, in his 2023 article “Uncertainty and Data Reconstruction” published in the Chinese Banking Industry magazine, Mongoose Think Tank’s Academic Committee Chairman Zhu Xiaohuang deeply discussed the issue of uncertainty in data. Zhu believes the essence of the world is uncertainty, causing randomness in human society and behavior. As data reflects the objective world and human behavior, it also carries uncertainty.
In his article “A Human-Centric Perspective on Data”, Mongoose Think Tank’s expert Hu Benli argues that all data subjectively reflects humans’ perspective on the objective world. He describes an ongoing cycle where humans generate large amounts of data while simultaneously being influenced by it. This inherent bias in data, Hu states, provides the fundamental rationale for needing data governance.
To address these issues of uncertainty and human-data relationships, Mongoose Think Tank have designed and proposed a method for reconstructing data based on these principles. Our approach categorizes data into types including historical data and marginal data, natural data and behavioral data, as well as deterministic data and random data. Building upon the elimination of random data, it assigns different adjustment parameters and weights to different types of data. Subsequently, Mongoose Think Tank applied and refined this methodology in real-world scenarios, culminating in the development of TDRU (Tools for Data Reconstruction on Uncertainty) version 1.0.
Introduction to TDRU
TDRU operates through six core processes, two sets of parameters, and seven collaborative tools. It can address both types of problems arising from uncertainty and adjust biases between humans and data. TDRU is not just a methodology but also practical as a tool. Its uniqueness lies in being the first tool specifically designed to tackle issues of data uncertainty, marking a significant advance over existing data processing methods. TDRU can be widely applied to various scenarios for predicting and measuring the likelihood of future events, making it suitable for major financial institutions, commercial enterprises, and global economic forecasts by world organizations. Additionally, Mongoose Think Tank are in the process of developing a data reconstruction intelligent assistant software that integrates TDRU with large language models.
TDRU is a foundational data tool based on the principle of uncertainty, primarily aimed at the pre-analytical stage of data analysis, namely data cleansing. Due to its foundational nature, the range of its applications is extensive, covering sectors such as economics, finance, and energy, among many others that require heavy use of data for modeling, analysis, and future prediction. Consequently, its client base is diverse, encompassing enterprises, financial institutions, and international organizations across the globe that need to employ data for forecasting. Additionally, due to the high correlation between uncertainty and risk management, TDRU offers unique value in the field of risk management. Overall, TDRU is not only a multi-functional and cross-sectoral methodology with international applicability but also a potent tool capable of solving a myriad of data quality issues, thereby enhancing the accuracy of future predictions.
TDRU Application Cases
Mongoose Think Tank applied TDRU to a personal credit scorecard of a listed bank. From the perspective of model evaluation parameters, before using TDRU, the model’s AUC (Area Under Curve, which measures the authenticity of the detection method; the closer AUC is to 1.0, the higher the authenticity; at 0.5, it has the lowest authenticity and no application value) was approximately 0.84, and KS (Kolmogorov-Smirnov, which evaluates the model’s risk differentiation ability; the greater the cumulative difference between good and bad samples, the greater the KS value and the stronger the risk differentiation ability of the model) was about 0.57. After using TDRU, the AUC increased to 0.88, and KS increased to 0.59. Importantly, these effects have been corroborated across multiple product lines.
Future
Currently, TDRU Version 1.0 has matured into a valuable tool for applications in macroeconomic forecasting, risk profiling and more. It has already obtained a software copyright and submitted an application for a patent.
Today, Mongoose Think Tank sincerely present this practical tool to the community at large, hoping to collaborate with various parties to expand the boundaries and possibilities of TDRU. Currently, Mongoose Think Tank can provide corresponding services related to TDRU, including consultation, training, and data processing. Mongoose Think Tank welcome inquiries from data enterprises, financial institutions, consulting firms, or any interested organizations. Please stay tuned for the future launch of TDRU Intelligent Assistant.
