Hello folks,
I've seen quite some articles on "the deep web" recently and I've read that it's like 550 times bigger than the actual "public web" so it kind of tinkled my curiosity.
The deep web is basically the web that is not referenced by search engines or not easily attainable.
There are multiples ways to hide a website from the eyes of search engines, TOR hidden services, Freenet, dynamic content, etc.
I mainly played with Freenet and here is the summary of my (short) exploration.
Since there are no search engines for the deep web. It is quite particular as how you surf on it.
The following has been experimented on Freenet but I'm sure you could apply some of it to other deep web services (TOR, etc).
The common way to surf on Freenet is to make use of indexes such, some a re unmaintained and other are maintained:
Linkageddon
In The Raw
Another Index
The Public Index
AFK index
Active Link Index
Uncensored Index
Entry Point
Then you can of course find freesite link through Google hacking:
Same on pastebin:
Use what is at your disposal on Freenet:
Sone (social network), boards, etc.
Then you can write custom spiders for instance.
There are also wikis, forums, irc, etc.
At first, I was quite disappointed since a lot of those freesites is shit, scumbags, paedophilia, etc. After searching a bit, you get to find good stuff:
XKCD
Fravia's archive pages of reverse engineering
Freenet Social Networking Guide
TI Calculator Signing Keys
AKG: Open Source Intelligence
Laputa Ebooks
Randomizer : Lockpicking
Without A Trace - A Forensics Manual for You and Me
Then there are emulation, abandonware, ebooks, etc, freesites.
I didn't find much freesites about hacking/reversing yet. If you have any interesting freesites, could be great to share ;).
Through boards and newsgroup, you can share files in Freenet. You could also find
long lost or forbidden software such as DeCSS (without this first DVD hack, actual software on Linux would still be unable to play DVD ;)), etc.
We can see that surfing the deep web is not as convenient and easy as the "public web" but it is possible with some patience.
In my usage, I found for now that there isn't anything on Freenet that I can't find on the internet.
The deepweb allow ways of expression, and in particular true for Freenet, that allows non censorship, complete freedom of information. This complete freedom of information comes with drawbacks for sure but you have the choice to ignore those weird/illegal/non ethical sites, nobody told you to go there in the first place.
Thus, I would mainly see Freenet as a mean to express and share information freely without censorship. It would be useful against repressing governments (China, etc) or in the current race for stupid laws (ACTA, SOPA, PIPA, HADOPI, etc, ... just die die die and kill those laws once and for all).
What is the Deep Web? A first trip into the abyss
The Deep Web
Wikipedia: Deep Web
I've seen quite some articles on "the deep web" recently and I've read that it's like 550 times bigger than the actual "public web" so it kind of tinkled my curiosity.
The deep web is basically the web that is not referenced by search engines or not easily attainable.
There are multiples ways to hide a website from the eyes of search engines, TOR hidden services, Freenet, dynamic content, etc.
I mainly played with Freenet and here is the summary of my (short) exploration.
Surfing the deep web
Since there are no search engines for the deep web. It is quite particular as how you surf on it.
The following has been experimented on Freenet but I'm sure you could apply some of it to other deep web services (TOR, etc).
Indexes
The common way to surf on Freenet is to make use of indexes such, some a re unmaintained and other are maintained:
Linkageddon
In The Raw
Another Index
The Public Index
AFK index
Active Link Index
Uncensored Index
Entry Point
Google hacking
Then you can of course find freesite link through Google hacking:
intext:"http://127.0.0.1:8888/usk@" OR intext:"freenet:usk@"
Pastebin
Same on pastebin:
http://pastebin.com/search?q=127.0.0.1%3A8888%2FUSK
Other ways of finding sites
Use what is at your disposal on Freenet:
Sone (social network), boards, etc.
Then you can write custom spiders for instance.
There are also wikis, forums, irc, etc.
What can we find there?
Anything since Freenet is a network distributed file system.
What is really interesting about Freenet though is that it is censor free, so even though you divulge information about something (like vulns, etc), it can't be taken down but by you.
What is really interesting about Freenet though is that it is censor free, so even though you divulge information about something (like vulns, etc), it can't be taken down but by you.
Freesites/Flogs/etc
At first, I was quite disappointed since a lot of those freesites is shit, scumbags, paedophilia, etc. After searching a bit, you get to find good stuff:
XKCD
Fravia's archive pages of reverse engineering
Freenet Social Networking Guide
TI Calculator Signing Keys
AKG: Open Source Intelligence
Laputa Ebooks
Randomizer : Lockpicking
Without A Trace - A Forensics Manual for You and Me
Then there are emulation, abandonware, ebooks, etc, freesites.
I didn't find much freesites about hacking/reversing yet. If you have any interesting freesites, could be great to share ;).
Filesharing
Through boards and newsgroup, you can share files in Freenet. You could also find
long lost or forbidden software such as DeCSS (without this first DVD hack, actual software on Linux would still be unable to play DVD ;)), etc.
Conclusion
We can see that surfing the deep web is not as convenient and easy as the "public web" but it is possible with some patience.
In my usage, I found for now that there isn't anything on Freenet that I can't find on the internet.
The deepweb allow ways of expression, and in particular true for Freenet, that allows non censorship, complete freedom of information. This complete freedom of information comes with drawbacks for sure but you have the choice to ignore those weird/illegal/non ethical sites, nobody told you to go there in the first place.
Thus, I would mainly see Freenet as a mean to express and share information freely without censorship. It would be useful against repressing governments (China, etc) or in the current race for stupid laws (ACTA, SOPA, PIPA, HADOPI, etc, ... just die die die and kill those laws once and for all).
References
The Deep Web
Wikipedia: Deep Web