The National Aeronautics and Space Administration (NASA) is set to develop a search engine specifically intended for the dark net. NASA is reportedly taking part in a project of the Defense Advanced Research Projects Agency (DARPA) to develop a system for marking content residing in the deep dark regions of the internet. The project is called Memex and is aimed at not only indexing content on the dark web but also identifying patterns of activity.
“When you do a simple web search on a topic, the results that pop up aren’t the whole story,” explains NASA.The Internet has a part called the “
The Internet has a part called the “deep web,” which is not indexed by search engines. This means that the usual search engines cannot show darknet pages in any of their search results. Only a search engine specifically created for the darknet can scour these content. One example of such a search engine is Grams. However, this search engine is specifically created to look for darknet marketplaces so it’s not really comparable to Google. What Memex aims to build is something close to what Google is capable of doing but on the dark net.
There are websites residing on the dark web, Onion sites in particular, that require a specific browser (the Tor browser) to be accessible. These websites may have mirrors that are being indexed by search engines like Google but don’t really provide access to the actual site. Darknet Heroes League, for example, can be accessed through a Google-indexed page with an “onion.to” URL suffix. This page, though, does not really grant access to the actual pages of the Darknet Heroes League site.
The Memex Project
Memex is a portmanteau of the words memory and index. The project intends to build a software that advances online search capabilities beyond what is currently available. In its official website, Memex is stated to have the goal of inventing “better methods for interacting with and sharing information, so users can quickly and thoroughly organize and search subsets of information relevant to their individual interests.”
“We’re developing next-generation search technologies that understand people, places, things, and the connections between them,” said NASA JPL principal investigator Chris Mattman in an interview with Daily Mail. “We’re augmenting web crawlers to behave like browsers. In other words, we are executing scripts and reading ads in ways that you would when you usually go online,” Mattman added.
The project aims to address the inadequacy of common search practices that don’t cover information in the deep web. It goes through all aspects of the web to generate data maps that can possibly gather clues about illegal activities.
Memex is actually not a new project. DARPA launched Memex back in September 2014, funding 17 teams to collaboratively create a software with new domain-specific indexing and search algorithms to be able to do the following:
- Improved content discovery
- Better information extraction
- Enhanced information retrieval
- User collaboration
- Extension of current search capabilities to the deep web, dark web, and nontraditional content
NASA works with Kitware and Continuum as one of the 17 teams to improve the Memex search technology. The team seeks to address complex search problems including the ability to thoroughly go over research papers, court documents, and information pertaining to human trafficking.
The Memex project is related to DARPA’s XDATA Big Data initiative. XDATA aims to process and analyze huge amounts of data for the benefit of defense, government, and civilian agencies. The resulting products under the Memex project are intended to become open source and free.
How Memex Helps
Generally, there are two main uses of the Memex search engine: fighting crime on the dark web and visual information search in the field of astronomy. As mentioned, many of the highlight features of the search engine being developed target online information and resources related to criminal offenses or illegal activities associated with the dark web. When it comes to the astronomy-related purpose, Memex is said to become useful for space missions that gather huge amounts of photos, videos, and imaging data with various instruments. The search engine’s image search capabilities can help scientists in analyzing the geological features of celestial bodies by enhancing the search for published scientific data.
Currently, there are four ways particularly mentioned on the Memex website through which the search engine is expected to be useful.
- Geoinformatics in Human Trafficking – The resulting search engine of the Memex project is not limited to traditional search parameters. It also has geospatial informatics capabilities that gather data involving victims of human trafficking.
- Facial Recognition – Memex comes with the ability to manage photos of potential terrorists and criminals and locate other places where exist on the web.
- Material Research – Memex is designed to be able to gather and analyze data from research papers and produce shared knowledge around specific issues or topics.
- Court Citations – To help identify human traffickers, the Memex search engine specifically crawls court documents online that can be linked with other resources to establish the identities of those involved in human trafficking.
There are five main features or applications to look forward to in Memex. They are as follows:
- ImageCat – Responsible for the analysis of images; ImageCat extracts EXIF metadata and texts embedded in an image via optical character recognition (OCR). Memex is capable of handling tens of millions of images.
- ImageSpace – This is the search engine’s ability to perform search and analysis over large numbers of images based on associated metadata and texts derived from OCR or based on an uploaded image.
- Memex Explorer – A pluggable framework, Memex Explorer is intended for doing crawls or searches that are domain specific. It also provides a unified interface for all Memex tools and comes with the ability to add links to a third-party web-based app.
- FacetSpace – This enables the investigation of huge data sets upon extracting and manipulating relevant facets.
- LegisGATE – This is a General Architecture Text Engineering application specifically developed to go over legislative resources.
What Memex Means for the Darknet Markets
Memex is bound to expose the dark web to a wider audience and to facilitate investigations to address crimes or illegal activities online. However, this is unlikely to alarm users of highly secure darknet markets. After all, Memex is just a search engine. It cannot go into private pages protected by logins. Darknet market users who carefully maintain their anonymity have nothing to worry about.
Memex can uncover a lot of things in the dark web but it does not defeat anonymity measures. It also does not break into the encryption strategies. PGP , is an example. Additionally, it is not designed to figure out the identities of bitcoin users, especially those that use bitcoin mixers and tumblers. Memex can facilitate investigations and efforts to address illegal activities online but not to the point that it can peer into private pages and encrypted correspondence.