* miscellaneous Do discussions about web-bots and how to deal with them happen on this "web bot" forum site?

[Guest Unregistered]

I've just spent a few minutes browsing through the forum and post titles on this site and can't figure out what this site is all about.

 

I operate a small-scale web server for the company I work for, and I've been examining our server's logs and blocking many IP's from entire countries that we don't deal with and have just started some interesting "counter-measures" for bots that are looking for wordpress .php files. I'm wondering if this is the site where such discussions take place, or is there some other web-forum for that?

[SumDumGuy]

Hi,

 

We discuss things of a nature that are reported by the finding of specific bots. Ones created with the sole purpose detecting anomalous linquistics for evaluation of precognitive leaks. I don't think you have come across any bots from us, directly. Perhaps you have backlinks emanating from here.

[Guest Unregistered]

> We discuss things of a nature that are reported by the finding of specific bots.

 

Are these specific bots operated by you?

 

Are their findings publically accessible?

 

> Ones created with the sole purpose detecting anomalous linquistics for evaluation of precognitive leaks.

 

Translation:

 

Ones created with the sole purpose of detecting language or content that may predict future events?

[SumDumGuy]

> We discuss things of a nature that are reported by the finding of specific bots.

 

Are these specific bots operated by you?

 

Are their findings publically accessible?

 

> Ones created with the sole purpose detecting anomalous linquistics for evaluation of precognitive leaks.

 

Translation:

 

Ones created with the sole purpose of detecting language or content that may predict future events?

 

Not operated by anyone of us here, no.

 

The translations of their findings are publicly accessible at halfpasthuman.com

 

Yes, the prediction of future events. Rather, the prediction of future language that may or may not manifest in the matterium. The current Report is outlined in our Public Thread, October ALTA 'Adult Supervision'

[SumDumGuy]

I guess I should also state, for clarity, that we are currently accepting members to the private side of the forum for those would would like to discuss the reports in a more in depth fashion. If you would like to join the private side you must first join the public portion of the forum here and send an email to, webmaster@webbotforum.com and he will get your permission set changed for you.

 

We were originally incorporated as a private forum but have been branching out trying to raise awareness. Thank you for your interest.

[Guest Unregistered]

I guess I misunderstood this site (web bot forum) to be focused on the discussion of the phenomena of web-bots (which I define as the autonomous or minimally-directed scanning of the internet by anonymous actors for content or to exploit vulnerable servers/devices).

 

This forum site does not seem to have that focus, and I still don't know what the point of this site is or the activities of it's participants nor the nature or ownership of the bots that perform actions that are the source of your interest.

 

Are we talking about the "majestic" web-bot? For which I block on-sight (based on the IP-ranges that it's http requests come from) ?

 

I see (and block) many strange non-human http requests to my server, and I'd be curious if I've ever seen the bot that is the focus of this site. What does this bot show in it's user-agent string? What IP-range(s) does this bot operate from?

[SumDumGuy]

I guess I misunderstood this site (web bot forum) to be focused on the discussion of the phenomena of web-bots (which I define as the autonomous or minimally-directed scanning of the internet by anonymous actors for content or to exploit vulnerable servers/devices).

 

This forum site does not seem to have that focus, and I still don't know what the point of this site is or the activities of it's participants nor the nature or ownership of the bots that perform actions that are the source of your interest.

 

Are we talking about the "majestic" web-bot? For which I block on-sight (based on the IP-ranges that it's http requests come from) ?

 

I see (and block) many strange non-human http requests to my server, and I'd be curious if I've ever seen the bot that is the focus of this site. What does this bot show in it's user-agent string? What IP-range(s) does this bot operate from?

 

No, we are not a site that is dedicated to the discussion of how bots work and operate. Though I'm sure you can find sites like that, both White and Black hatted.

 

I, nor are anyone within these confines, that I am aware of, am not privy to exact nature of the bots whose findings we study. As such, information you are looking for is unknown to anyone outside of a select few.

 

cheers

[ChiliCheeseFries]

"bots" or "crawlers" basicly are scripts that run on servers which visit a list of websites. They upload content, usually words, to their storage and process this in whatever way is determined by the programming. Google and other search engines do this incessantly.

 

Search terms: search engine crawler program

 

Here is a basic how to article: http://www.makeuseof.com/tag/build-basic-web-crawler-pull-information-website/

 

Sorry if this is too elementary for your interests. GitHub probably has a lot of open source projects available if that is of interest.

[Guest Unregistered]

> Sorry if this is too elementary for your interests. GitHub probably has a lot of open source projects available if that is of interest.

 

I'm not looking to build or operate a web-bot, nor to know the details of how they are coded.

 

I'm looking for discussions on the "real-time" observations of others like me who observe, through the log files of their web servers, the various "actors" who are accessing their site, and collectively putting a name to individual actors or group them according to the goals of their activity.

 

Many people who operate SMTP (email) servers do this in terms of knowing which sending IP's to block or to accept in terms of email reception, and there are various lists (spamhaus being probably the most well known) in terms of assigning a reputation or score as to whether or not a mail server should accept email from a given IP (ie - is the email likely to be spam? If yes, then reject the SMTP connection request).

 

A phenomena that I've been seeing for a while are blind attempts to access various word-press files (I say blind because they are just assuming these files exist). Files such as "wp-login.php". Since I have no such files on my server, this results in a "404" error being handed to the bot asking for the file. What I've done lately is create a very large bogus file with that name, and the bots are now getting that file, and I hope they enjoy it. A new twist is that I'm performing a re-direction on that file so the bots are now following the re-direction to another server for that file, and I've tested this and it appears to be working (they are following the re-direction). This will off-load the network traffic from my server to some other server where the bots can enjoy downloading possibly a several gb-sized file because they had the gall to waste my server's time.

[jdurand]

I've been seeing 404 errors like that for MANY years (probably back into the 1990s) on the server I run for my business. They're nothing new.