Can I Get List Of All Websites Running In World? Domain Names?

September 8th, 2009 by Cheap Web Hosting Leave a reply »

Here I tried to do with NSLOOKUP > ls -d
I have to put domain name over there. I need a list of all websites available on the internet.
I thought logical way would be fetching all forward lookup zones in any root server, so I’ll get list of all available domain names. I don’t know if I’m correct.
Can You help me in this?

Advertisement

2 comments

  1. Jack says:

    This is an awesome question. I was trying to find the answer after reading this but I don’t know how it’d be possible.
    I don’t think you’d be able to get them through cmd.
    I know CERN invented the internet though i’m not sure if they entirely run it to this day.
    There would have to be a way cause when you check for available domains, you get results on what’s taken and what’s not. So there has to be a database somewhere. Maybe googling entire domain database:
    no i just tried a few things. not sure.
    if you find the answer, you should email it to me at jack@vicariousconcept.com

  2. rainofka says:

    well u could do u have the resources and cycles for this…. nslookup is a defunct tool,,, i would use the host command,, ur best bet would be to work out a way to recursively query dns from root down,, actually ur best bet would be to iterate through every logical IP address in a loop,,, doing reverse dns queries for each $IP in that loop (friggin huge loop),,,u would then for intergrity (&& every site will not have a reverse entry so this is not even accurate) need to probably actively query a dns server or run a whois on the sites,,, u will need to format this data in some way like use awk or cut,, or most likely both and this will be such a wopping amount of data ur only chance of reviewing it logically would be to have it stored to a database,,, this all is actually very easy one thing to note is this type query on root dns will certainly bring government authority attention,,, another thing u can do is crawl the internet,,, or even crawl the crawlers,,, u can maybe write a perl script or bash that crawls googles search pages,,, and maybe uses some regex to produce websites allinurl:.`com|net|et… ,, this even u will need to watch as u dont want to be crawling google to much either,,, BUT heres a really cool site i know about;; this is rumored to contain a copy of almost every site that ever existed IN THE PAST and no longer exists try it out http://www.archive.org/index.php