Google & the InterNet

Ray Montee (RIP) · Post by **Ray Montee (RIP)** » 25 Feb 2005 4:46 pm

A recent check on Google.....and I find that it contains references to all the pages within the Jerry Byrd Fan Club web site.
I'm grateful for this but can't help but wondering "WHO" did this? It's great to have that "coverage" but does someone "at Google" get paid for doing that or do various interest parties merely submit things of that nature.
Just an old man's curiosity......

Rick Aiello · Post by **Rick Aiello** » 25 Feb 2005 5:13 pm

Search engines (Google, AltaVista, etc) have Web Crawlers ...

Programs that continually "crawl" from site to site ... downloading documents and following links from page to page.

I'm sure someone will jump in and explain it better ...

------------------

Aiello's House of Gauss


My wife and I don't think alike. She donates money to the homeless and I donate money to the topless! ... R. Dangerfield

Joey Ace · Post by **Joey Ace** » 25 Feb 2005 5:15 pm

As Rick said, it's an automated process.
The search engines have "web-bots". They are programs the search every site on the net looking for key words.

I'm amused when I search for my name I find a Pro-Wrestler in the UK with the same name. I haven't yet found his picture, but he has lost every match I found a reference to.

Maybe he should use a Steel Guitar as a gimick. Hitting someone over the head with a D-10 could win him a few matches.[This message was edited by Joey Ace on 25 February 2005 at 05:19 PM.]

Jim Phelps · Post by **Jim Phelps** » 25 Feb 2005 5:34 pm

Usually, someone has to submit the website's address in the first place, then it kinda spreads out from there.... I submitted my site to the major search engines years ago and now I find it's listed in search engines I've never even heard of. Many of them share or possibly sell or trade data. Depending on the search engine, it may list every page or not. Most of the big ones are free, but some are trying to make a buck at it now. I've never paid a cent to a search engine and there aren't many I'm not in. A tip, don't use the companies who send you email spam and claim something like "we submit your site to 4000 search engines" for some price. They use an automatic submission program, and almost all the search engines reject this kind of submission. You gotta do it one by one by hand. Then, as Joey said, once your site is listed, the "robots" crawl the web for any updates to the pages they've listed. Their schedules of crawling for updates varies, it may be anything from weekly to every few months.[This message was edited by Jim Phelps on 25 February 2005 at 05:37 PM.]

Perry Hansen · Post by **Perry Hansen** » 25 Feb 2005 7:39 pm

Ray. Go to Google and type in your name and see what comes up

Jim Smith · Post by **Jim Smith** » 25 Feb 2005 9:12 pm

I've found a "few" links to my name!

Jason Odd · Post by **Jason Odd** » 25 Feb 2005 11:42 pm

Geez Ray, you get lots of hits.. cool.

I rarely pop up, although I do have someone in the same name who does tattooing, I was somewhat bummed to know that I'm not one of a kind!

Actually it's not all automated, there is genre and other things taken into the web search, in fact I went for a job with Yahoo some years back.
The job was to cruise the web and crunch statistics and catergorize info to improve web searches.
They are after all businesses, some people with pages with tiny servers do not show up on searches, or if they do.. it's after 30 or more pages, no matter how distinct their name is.
I know this from experience, is it some sort of marketing thing?..ie the server is too small to pay google and other servers to be listed for hits.

Has anyone else run across this?

richard burton · Post by **richard burton** » 25 Feb 2005 11:53 pm

Some actor guy seems to take precedence over me

Jim Phelps · Post by **Jim Phelps** » 26 Feb 2005 1:54 am

Jason, I don't believe that any search engine ignores "pages on tiny servers" or would even know whether it's on a huge or tiny server in the first place.

Your job crunching data and catagorizing info may have been not automated and therefore subject to personal discretion, but catagorizing info is not the same as deciding who gets listed and who doesn't, or where in the results the listing shows up.

As far as I know, there is some personal input into what sites are listed, most of the search engines have a disclaimer which says they do not promise or guarantee that every site submitted will be listed, apparantly that is subject to personal approval, at least with some search engines.

Where the site will be placed in the results listing, is automatic and depends on how many other sites with similar names are listed, what kind of keywords are in the META tag for keywords, how many times the pages have the word that was searched on to find the site.

For example, if my site is about Pedal Steel and that word is in the keywords META tag, and I've used that word in html text many times on each page, my pages will show up in the listings before another site that only has Pedal Steel once or twice on his page.

The different search engines are not all set-up exactly the same, that's why some engines will show some sites on page 1 of the results and some others will show the same site much later, even when the same search word was used.

[This message was edited by Jim Phelps on 26 February 2005 at 01:56 AM.]

Jim Peters · Post by **Jim Peters** » 26 Feb 2005 4:25 am

Google is set up so that the first page that shows up is the one that gets the most hits. It is their policy that nothing influences what shows up first, other than number of hits.
Anyone could pay to have their website show up in Google, but it would be listed as a sponsored link.
I don't show up when Googled,but my wife's picture is usually 2nd or 3rd with an image surf. It really freaked her out!JP[This message was edited by Jim Peters on 26 February 2005 at 04:27 AM.]

Joey Ace · Post by **Joey Ace** » 26 Feb 2005 5:20 am

Most of the hits on my web page come from Google. I'm now at the top of their list.

Being first on their list brings more hits.

It's a circle! "The Hits Keep a-Coming".

Jim Jasutis · Post by **Jim Jasutis** » 26 Feb 2005 6:14 am

Dad read an article the other day where someone was complaining that the first reports on a google search of a name always gave the most negative or sensational information. I guess this could be because as someone esls mentioned the most hit sites were listed first, and I guess it is human nature that this would be the negative sites.

Bobby Lee · Post by **Bobby Lee** » 26 Feb 2005 9:11 am

Two additional factors:

1. People who use the Google toolbar have the option of letting Google "spy" on their browsing. It's turned off by default (as it should be!), but if you want to influence Goggle ratings for the sites you visit, you should turn it on.

2. Goggle's web crawlers count links to your site. If a lot of other sites link to you, you will get a higher ranking on Google.

I don't believe that the meta keywords tag has as much influence as people think, but it doesn't hurt to add it to your blank page template.

Jim Phelps · Post by **Jim Phelps** » 26 Feb 2005 1:49 pm

Carter's website and Brad's Page of Steel are both listed ahead of the Steel Guitar Forum, when you search Google on "pedal steel". They're both great sites, but get more hits than the Forum? I can't believe that. b0b also has huge amount of links here, and I'd bet that more website have links to the SGF than the Carter or Brad's sites.

My own site has been listed for over 6 years and I get a LOT of hits, I come up number 6 (today) in the listings at Google, yet I'm sure that other sites get plenty more hits than I do.

b0b's right about inbound links ("PageRank") contributing greatly to the ranking order, but there are also a lot of other factors, from a page on how Google works: "Apart from Pagerank, there are *hundreds* of factors that Google takes into account while ranking web pages."

How about all of us with websites put in a link to the Forum and see if we can get it No. 1 in the rankings? Should have a link anyway. I've had a link to the Forum in my site for years.

You can get more info here: http://www.searchengineethics.com/google_overview.htm

[This message was edited by Jim Phelps on 26 February 2005 at 02:36 PM.]

John Fabian · Post by **John Fabian** » 27 Feb 2005 4:05 am

The goal of search engine placement is to be on the first page (top 10). It's nice if you are #1 but if you are among the first 10, the description of your site probably influences the vistor more than the actual ranking.

I would think that most people looking for pedal steel or steel guitar (or some form of those phrases) are more interested in buying one or seeing one. But, either way, all three sites feed each other.

All 3 of the sites Jim Phelps mentions have great information and contain totally different content.

b0b's site is easily the most heavily visited and improving his ranking will probably do almost nothing to improve the number of visitors.

------------------
John Fabian
Carter Steel Guitars

www.steelguitar.com
www.steelguitarinfo.com
www.carterstarter.com

Mark Ardito · Post by **Mark Ardito** » 28 Feb 2005 7:34 pm

If you want a web-bot or web crawler to avoid harvesting results on a website you would include that page in a file called "robots.txt" and place that file in your www root directory.

b0b - For example, if you wanted to stop web-bots from harvesting information from your user profile pages you would make the file robots.txt and include the following:

User-agent: *
Disallow: /cgi-bin/ubbmisc.cgi

If you wanted to stop a specific BOT/Crawler then you would enter in that specific name:

User-agent: googlebot
Disallow: /cgi-bin/ubbmisc.cgi

All bots must recognize the robots.txt or else you can register a complaint against them. Any raw weblog will be able to show what bots are not following your robots.txt file instructions.

Cheers!
Mark

------------------
Sho~Bud Pro I, Fender D-8 (C6&E13) http://www.darkmagneto.com
http://www.arditotech.com