Difference between revisions of "Documentation"

From DeepMAC Wiki
Jump to: navigation, search
(Created page with "== Documentation == Project documentation for DeepMac is still fragmented and uneven. For now, this section will contain previously written information that has appeared in v...")
 
(→‎Data)
Line 10: Line 10:
  
 
The heart of the DeepMac project is the data collected, analyzed and organized. In order to do this, the Standards above are used to guide the specific formats for data interchange and storage.
 
The heart of the DeepMac project is the data collected, analyzed and organized. In order to do this, the Standards above are used to guide the specific formats for data interchange and storage.
 +
 +
# Sources
 +
## IEEE
 +
## DeepMac project participants
 +
# Stages
 +
## Gathering
 +
### Cleaning / Normalizing
 +
### Storing
 +
## Analysis
 +
### Retrieving / Storing / Modifying
 +
### Verifying
 +
## Presentation
 +
### Indexing
 +
### Searching
 +
### Normalizing
 +
 +
All of these stages of data mean a consistent way to containerize it, with a format that's widely in use and easy to work with. Since DeepMac is tasked with long-term goals we also want a clear and auditable trail for when data entered the project and when it was changed.
 +
 +
DeepMac thus does not "delete" data.
 +
 +
Instead, *all* changes to data related to MAC addresses and authoritative registrations of same are to be recorded in journals. Journals are to be added to but never removed from, nor deleted. This does mean that differentiation must be done from the journals to derive meaningful and complete information about a MAC address/OUI. Yet the ability to view historical activity in the process of IEEE registry changes alone makes this worthwhile.
 +
 +
==== DeepMac Record Format ====
 +
 +
Each OUI that has ever been documented from IEEE sources will be given its own journal. All data about the OUI will be recorded as entries in the journal using DeepMac Records (aka dmRecord),  which contain both the information to record as well as metadata regarding the entry itself.
 +
 +
* dmRecord must always contain this data:
 +
## "DeepMac" flag, to indicate the dataset is to be treated as a dmRecord
 +
## Unix timestamp
 +
## Date of event/action
 +
## Type of event/action
 +
## OUI
 +
## OUI Size
 +
## Source?
 +
* dmRecord will contain other data depending on the dmRecord type:
 +
## IEEE Registry
 +
### Organization name
 +
### Organization address
 +
### Organization country
 +
## Metadata
 +
### MAC start
 +
### MAC end
 +
### Confidence level
 +
### Media type (physical layer)
 +
### Device type
 +
### Device model
 +
### Note
 +
### Wiki link
 +
* DeepMac currently only supports JSON as the container for dmRecords
  
 
=== Code ===
 
=== Code ===
  
 
DeepMac's code has been open source from the beginning, but is not married to a specific platform. The long-term goals necceistate this: The project may need to move to other languages, digital platforms or re-vamp entirely as technologies change.
 
DeepMac's code has been open source from the beginning, but is not married to a specific platform. The long-term goals necceistate this: The project may need to move to other languages, digital platforms or re-vamp entirely as technologies change.

Revision as of 10:22, 11 August 2017

Documentation

Project documentation for DeepMac is still fragmented and uneven. For now, this section will contain previously written information that has appeared in various blogs.

Standards

DeepMac relies on standards developed for the project. This section attempts to stipulate the most critical of these in a succinct manner.

Data

The heart of the DeepMac project is the data collected, analyzed and organized. In order to do this, the Standards above are used to guide the specific formats for data interchange and storage.

  1. Sources
    1. IEEE
    2. DeepMac project participants
  2. Stages
    1. Gathering
      1. Cleaning / Normalizing
      2. Storing
    2. Analysis
      1. Retrieving / Storing / Modifying
      2. Verifying
    3. Presentation
      1. Indexing
      2. Searching
      3. Normalizing

All of these stages of data mean a consistent way to containerize it, with a format that's widely in use and easy to work with. Since DeepMac is tasked with long-term goals we also want a clear and auditable trail for when data entered the project and when it was changed.

DeepMac thus does not "delete" data.

Instead, *all* changes to data related to MAC addresses and authoritative registrations of same are to be recorded in journals. Journals are to be added to but never removed from, nor deleted. This does mean that differentiation must be done from the journals to derive meaningful and complete information about a MAC address/OUI. Yet the ability to view historical activity in the process of IEEE registry changes alone makes this worthwhile.

DeepMac Record Format

Each OUI that has ever been documented from IEEE sources will be given its own journal. All data about the OUI will be recorded as entries in the journal using DeepMac Records (aka dmRecord), which contain both the information to record as well as metadata regarding the entry itself.

  • dmRecord must always contain this data:
    1. "DeepMac" flag, to indicate the dataset is to be treated as a dmRecord
    2. Unix timestamp
    3. Date of event/action
    4. Type of event/action
    5. OUI
    6. OUI Size
    7. Source?
  • dmRecord will contain other data depending on the dmRecord type:
    1. IEEE Registry
      1. Organization name
      2. Organization address
      3. Organization country
    2. Metadata
      1. MAC start
      2. MAC end
      3. Confidence level
      4. Media type (physical layer)
      5. Device type
      6. Device model
      7. Note
      8. Wiki link
  • DeepMac currently only supports JSON as the container for dmRecords

Code

DeepMac's code has been open source from the beginning, but is not married to a specific platform. The long-term goals necceistate this: The project may need to move to other languages, digital platforms or re-vamp entirely as technologies change.

Retrieved from "http://wiki.deepmac.org/index.php?title=Documentation&oldid=13"