Search Engine Optimization > Web Development > Header modifying web proxies
Header modifying web proxies
Posted by James Taylor on July 8th, 2003

I have a site which doesn't get much external traffic but does get
viewed regularly by internal staff. In order to get a less distorted
picture of which parts of the site are generating external interest,
I would like to find some way to filter the internal hits out of
the log. Sadly, I can't distinguish upon IP address because their
office has a shared dial-on-demand connection with an ISP that
allocate addresses dynamically. The best idea I've had so far is
to configure all the internal browsers to go through a proxy
running on the LAN<->Internet gateway machine. The proxy software
would then either modify or preferably add an HTTP header so that
those hits can be distinguished in the log stats. The logging
facility records all non-standard headers so these would show up.

Can anyone recommend some inexpensive/free proxy software (for
Windows NT) that allows headers to be added or modified? Thanks.

Failing that, I could write my own proxy in Perl. I've not had
to do this before so I would be grateful for any hints & tips,
proxy writing primers, useful Perl modules, example code, etc.

Alternatively, you may have a better way of doing this, such as a
means of configuring MSIE itself to output different HTTP headers.
Perhaps the solution will be to insist everyone uses Opera
(assuming Opera can be configured in this way) although I suspect
there will be too much resistance to that.
Do you have any other ideas?

Thanks again.

--
James Taylor, Cheltenham, Gloucestershire, UK. PGP key: 3FBE1BF9
To protect against spam, the address in the "From:" header is not valid.
In any case, you should reply to the group so that everyone can benefit.
If you must send me a private email, use james at oakseed demon co uk.

Posted by James Taylor on July 9th, 2003

In article <85d6glm7ck.fsf@persephone.doilywood.lan>,
Richard Watson <tinnedmeat@doilywood.org.uk> wrote:
How easy is Apache to install on Windows and configure as a
proxy? I must confess to never having done either of those
things let alone in combination. Do you have any tips for
the proxy config neophyte?

Interesting idea but wouldn't that involve the public server
having to access data on the internal server in order to
generate the stats? That may not be impossible but it
doesn't sound ideal, or particularly secure for that matter.

Yes, when you include those benefits, it sounds like an
excellent idea. Thanks.

--
James Taylor, Cheltenham, Gloucestershire, UK. PGP key: 3FBE1BF9
To protect against spam, the address in the "From:" header is not valid.
In any case, you should reply to the group so that everyone can benefit.
If you must send me a private email, use james at oakseed demon co uk.


Posted by Richard Watson on July 9th, 2003

James Taylor <spam-block-@-SEE-MY-SIG.com> writes:

I must admit that I've never set up apache on windows, but I'm told by
people who have that it's not difficult.

Start here http://httpd.apache.org/docs/mod/mod_proxy.html - if you're
at all used to apache it should be fairly simple. Simple set up the
proxy on some port (maybe even 8080) and point everyone's machine at
it.

Of course it will then be proxying for all outgoing web stuff, but
that's not a bad idea.

That depends on how things are set up and where you want to see the
stats from. I'm not sure myself if this is such a great idea, but its
certainly worth considering.

I can't say I've tried this out yet but probably I'd just rely on some
custom logging format on the public server to provide enough there to
grep for to ensure that you can eliminate anything that came via the
proxy in your stats.

--
Richard Watson
http://www.opencolo.com/
High Value Colocation

Posted by Santiago Zawojski on July 9th, 2003

James Taylor <spam-block-@-SEE-MY-SIG.com> wrote in
news:ant090002345fNdQ@oakseed.demon.co.uk:

Start > Run > regedit ?

Why is that not what you want?

Posted by Jim Ley on July 9th, 2003

On Wed, 9 Jul 2003 01:36:02 +0100, James Taylor
<spam-block-@-SEE-MY-SIG.com> wrote:

Lots of things are tweakable in the registry for IE.

Nah, it works for all, although you may need to change it if they
install a new IE since that may overwrite the strings (use MS only
installs to avoid this - or customise your own install containing the
new user agent)

Jim.

Posted by James Taylor on July 9th, 2003

In article <Xns93B35A38B61DAauto352683WOMBAT@216.166.71.238>,
Santiago Zawojski <auto352683wombat@hushmailkangaroo.com.invalid> wrote:
If it works on every version of Windows then, yes,
it *is* what I want. Thanks.

--
James Taylor, Cheltenham, Gloucestershire, UK. PGP key: 3FBE1BF9
To protect against spam, the address in the "From:" header is not valid.
In any case, you should reply to the group so that everyone can benefit.
If you must send me a private email, use james at oakseed demon co uk.


Posted by James Taylor on July 9th, 2003

In article <851xx0mcew.fsf@persephone.doilywood.lan>,
Richard Watson <tinnedmeat@doilywood.org.uk> wrote:
That's very helpful. Thanks. The next hurdle will be persuading
them to find a moment for me to set it up and test it.

Yes, I agree, and I've already had to put custom logging in place as
a result of the web space provider not providing any logging as standard.
The question of whether to use this solution or the regedit of MSIE's
User-Agent will largely depend on how comfortable they are about
me experimentally installing foreign software on their main server. ;-)

--
James Taylor, Cheltenham, Gloucestershire, UK. PGP key: 3FBE1BF9
To protect against spam, the address in the "From:" header is not valid.
In any case, you should reply to the group so that everyone can benefit.
If you must send me a private email, use james at oakseed demon co uk.


Funbolt.com - Entertainment portal, wallpapers, sexy celebs