Dan Luu
@danluu
If the site's too large to wget and it doesn't have a custom index, I frequently can't find the page. Large commercial sites, like Twitter, sometimes build complete indexes, but it's a non-trivial effort to index something that's even 1/100th the size of Twitter, so most don't.
