Machine learning has become the cornerstone of big data

Machine learning is almost everywhere, and even if we don't specifically call them, they often appear in big data applications. I used to describe some typical big data use cases in my blog. In other words, these applications can provide the best results in the "extreme situation." In the final part, I also mentioned the combination of tapping byte-level data capacity, real-time data speed, and/or multi-structured data diversity.

At the time, I also listed a list of applications and deliberately avoided machine learning analysis during the collection process. The main reason is that, although machine learning is not a major but a common tool in these use cases, it is not a use case in itself. In other words, they are not a special application domain formed by their own power. For the same reason, I did not list schema design, metadata management, or data integration like the big data use case. But like machine learning, they also make their own contribution to the value of big data analytics applications.

The contribution of machine learning to the return on investment of big data applications is mainly reflected in two aspects: one is to promote the prolificacy of data scientists; the other is to find some neglected programs, and some programs have even been ignored by the best data scientists. . These values ​​come from the core function of machine learning: that is, the analysis algorithm can learn the latest data without human intervention and explicit procedures. The solution allows data scientists to create a model based on typical data sets, and then use algorithms to automatically summarize and learn these examples and new data sources.

In many cases, machine learning is the best investment return for big data innovation. The investment in machine learning can deepen any big data case customized to the company. This is because machine learning algorithms are becoming increasingly efficient in terms of capacity, speed, and type (ie, the 3 V characteristics of big data). As Mark van Rijmenam said in a recent article on machine learning: "The more data you process, the better the algorithm will show its advantages." He believes that it includes voice and facial recognition, clickstream processing, and search engines. Many machine learning applications, including optimization and recommendation engines, may be described as sense-making analytics.

Analytical methods require constant monitoring of user semantics, content, and importance that are inferred from the data stream. In order to support the automation of the idea, machine learning algorithms must often deal with something extremely complex. This includes the hidden semantic classifications that make up an object or environment. This requires real-time collection of the overall meaning through a variety of different data streams. These data streams must include different objects such as data, video, images, voice, expressions, actions, geographic information, and browser clicks. The meaning that is automatically extracted from these data streams through machine learning may be mixed with cognitive, emotional, sensory, and volitional features.

In order to find clues in these materials, "deep learning" has become an important tool in the machine learning instruction system of big data scientists. As Van Rijmenam puts it, deep learning using neural networks can help to extract perceptual capabilities from these data streams because these data streams may involve hierarchical arrangements that form the semantic relationships between objects. “In-depth learning can break the gap between the components with different characteristics in the data and use these features to find out different combinations of features to find out what they see or what they are doing,” van Rijmenam said.

Obviously, machine learning is a fundamental tool for creating an environment that can sense and handle dynamic distributed solutions. The ability of humans to detect and respond to real-time threats and terrorist activities, natural disasters, hurricanes, and other threats depends on the automatic screening, classification, and correlation of information in vast amounts of data. Without this capability, humans are in danger of being "drowned" in the oceans of big data.

36 Big Data Knowledge Atlas: About Machine Learning

Machine Learning (ML) is a multi-disciplinary field and involves many disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. Specializing in how computers simulate or realize human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance.

It is the core of artificial intelligence and the fundamental way to make computers intelligent. Its application covers all areas of artificial intelligence. It mainly uses induction, synthesis rather than deduction.