1. Building a Project¶
NoviLens is a diverse set of tools that allows the user to explore their data in an interactive fashion, allowing the user to drill down as deep as they desire. The system is designed to be flexible, working with both structured (databases) and unstructured (text) data. The underlying principle behind Lens is the “item” and the “feature”. The item is the “thing” or question being asked. The feature is a term or fact used to describe the item. An item may have many features. Sometimes the user needs to collect features from many different sources or tables, thus the capabilities for data fusion, changing data types, and establishing data relationships. Words are ambiguous. Lens uses Dictionaries to arrange words into topics. These topics help with machine learning or can be used as features.
Users are unique, they process data uniquely therefore Lens Dashboards are designed to be configurable by the individual user to meet their style of data interpretation. These Dashboards can be shared or copied; tailored for the individual or the activity.
Because the concept of item and feature are carried through the design of Lens, machine learning has been built into the system. Using these tools, the user interacting with the machine, can perform a wide array of machine learning tasks to classify or predict based on the data present in their dashboards.
Finally, you can’t break the system. You may clog it up but it can be reset. Experiment with dashboards to see how relationships work and to determine what provides the best view of your data. Most importantly, frame your question by understanding the facts you need to answer your question. By using this approach, you will get the most benefit from NoviLens.
1.1. The process of building a project consists of the following:¶
Creating a project name
Bring data into the system
Forming relationships between data
Developing Data Exploration Tools (widgets)
Expanding Analytics with Natural Language Processing and or Machine Learning
1.2. Creating the project and bring data into the system.¶
The first step in building a project is creating that project in the system. This is accomplished by creating a project name and associating the primary data table with the project. This primary data table can be a c.s.v. file, a table from a database, or files associated with the file/web crawler.
1.3. Understanding Data Types¶
Not all data is represented in the same manner even though it may look the same on the screen. Numbers may be floats (decimal points) or integers. Time series may look like text. All c.s.v. imports will be represented as text values. This becomes an issue when developing relationships between data tables and how data interacts with widgets - text acts differently than numbers.
The tools in the data importer allow the user to recast data to different types provided the format of the data is compatible with the new type. A float can’t be converted to an integer but an integer can be converted to a float. Text can be converted to numeric values. If the text contains a decimal point, it’s a float.
1.4. Establishing Relationships in Data¶
Different data tables on a topic can be linked by a common data field in the data table. For instance a state name, a customer i.d., or a word (advanced feature). This is handled as either a primary-foreign key relationship or a many-to-many relationship in the application.
1.5. Mapping Latitude and Longitude¶
The tables section is where the user configures geographic points for the mapping widget. Points are the representation of latitude and longitude on the map. Points can be configured topic::represent relative areas such as 500 meters, 5000 meters, etc.