FAQ

If the Log File Analyser says your ‘licence key is invalid’, then it's because it has been entered incorrectly. This can be seen under 'Licence > Enter Licence Key' in the interface.

Licence keys are displayed on screen when you check out, sent in an email with subject "Screaming Frog Log File Analyser licence details" and available at any time by logging into your account.

Before contacting us, please view your licence details and copy and paste them into the correct field - they are not designed to be entered manually. Please follow our tips below on common issues encountered.

• Ensure you are using the username we provided for your licence key, as this isn't always the same as your account username and it's not your email address. This is by far the most common issue we see
• Copy and paste the username and licence key, they are not designed to be entered manually.
• Please also double check you have inserted the provided ‘Username’ in the ‘Username’ field and the provided ‘Licence Key’, in the ‘Licence Key’ field.
• You are entering a SEO Spider licence into the Log File Analyser.
• You are entering a Log File Analyser licence into the SEO Spider.

If your licence key still does not work, then please contact support with the details.

Licences are individual per user. A single licence key is for a single authorised user. If you have five users using five copies of the Screaming Frog Log File Analyser software, you will require 5 separate licences. Please see section 3 of our terms and conditions for full details.

Yes. The licence allows you to install the Log File Analyser on multiple computers. Licences are individual per user. Please see section 3 of our terms and conditions for full details.

When the licence expires, the Log File Analyser returns to the restricted free lite version. You will be able to open old projects, but you won't be able to add to them or create new projects.

The Log File Analyser stores the licence in a file called licence.txt in the users home directory in a ‘.ScreamingFrogLogfileAnalyser’ folder. You can see this location by going to Help->Debug and looking at the line labeled “Licence File”. Please check the following to resolve this issue:

• Ensure you are able to create the licence file in the correct location.
• If you are using a Mac, see the answer to this stackoverflow question.
• If you are using Windows is could be the default user.home value supplied to Java is incorrect. Ideally your IT team should fix this. As a work around you can add: -Duser.home=DRIVE_LETTER:\path\to\writable\directory\ to the ScreamingFrogLogfileAnalyser.l4j.ini file.

No, the Screaming Frog SEO Spider is a separate product to the Log File Analyser. They have different licences, which will need to be purchased individually.

If you have just purchased a licence and have not received your licence, please check your spam / junk folder. Licences are sent immediately upon purchase. You can also view your licence(s) details and invoice(s) by logging into your account. Please also check your payment method. If you have paid via an e-cheque, then the licence will only be sent when it has cleared. Paypal explains this as well.

A licence removes the 1,000 line log file limit and allows you to have multiple projects saved at a time. We also provide support for technical issues related to the Log File Analyser for licensed users.

Licences are individual per user. If you have five members of the team who would like to use the licensed version, you will need five separate licences for each user.

You can read our EULA here.

Simply click on the ‘buy a licence’ option in the Log File Analyser ‘licence’ menu or visit our purchase a licence page directly.

You can then create an account and make payment. When this is complete, you will be provided with your licence key to open up tool and remove the log file and project limits. If you have just purchased a licence and have not received your licence, please check your spam / junk folder. You can also view your licence(s) details and invoice(s) by logging into your account.

Please note, if you purchased via bank transfer or PayPal invoice you will not have an account. An account is only created when you checkout via our online purchase system. If you have lost any of your details you can contact us for this information.

You simply login to your existing account and purchase another licence upon expiry.

Please don't purchase a licence pre-expiry as it will not extend your existing licence expiry. Updated licence details will be provided upon purchase to continue using the tool.

Licences do not currently auto renew, so if you do not want to renew your licence you need to take no action as you will not be automatically charged.

As standard you download the free version of the tool. However, without a licence the Log File Analyser is limited to analysing 1,000 log file event lines in a single project.

For £99 per annum you can purchase a licence which removes the 1,000 log file event limit and number of projects you can create.

A licence is required per individual using the tool. When the licence expires, the Log File Analyser returns to the restricted free version.

We only provide discounts on bulk purchases of 5 or more licences at a time. We do offer discounts at any other time.

We simply keep the price as competitive as possible for everyone.

Please see the Log File Analyser licence page for more details on discounts.

We accept most major credit and debit cards and PayPal.

The price of the SEO Spider is in pound sterling (GBP), but it can be purchased worldwide.

If you are outside of the UK, please take a look at the current exchange rate to work out the cost. The automatic currency conversion will be dependent on the current foreign exchange rate and perhaps your card issuer, so we are not able to give an exact fee (other than in GBP).

We do not accept cheques (or checks!) as payment.

Yes, if you are not in the UK. To do this you must have a valid VAT number and enter this on the Billing page during checkout. Enter your VAT number in the 'Tax ID' field, and then remember to click 'Verify'. Your VAT number will be checked using our External Tax Validation service and VAT removed if it is valid. You can check your VAT number using the VIES VAT validation system here. The VIES service can go down from time to time, please see here for your options if this happens.

Unfortunately we cannot refund VAT once a purchase has been made.

Unfortunately we do not get to see why a payment has been declined. There are a few reasons this could happen:

• Incorrect card details: Double check you have filled out your card details correctly.
• Incorrect billing address: Please check the billing address you provided matches the address of the payment card.
• Blocked by payment provider: Please contact your card issuer. Screaming Frog does not have access to failure reasons. It’s quite common for a card issuer to automatically block international purchases.

The most common of the above is an international payment to the UK. So please contact your card issuer and ask them directly why a payment has been declined, and they can often authorise international payments.

Alternatively, you can try a different card, PayPal, or you can contact us via support and pay via bank transfer.

Absolutely! If you are not completely satisfied with the Log File Analyser you purchased from this website, you can get a full refund if you contact us within 14 days of purchasing the software. To obtain a refund, please follow the procedure below.

Contact us via lfasupport@screamingfrog.co.uk or support and provide the following information:

• Your contact information (last name, first name and email address).
• Your order number.
• Your reason for refund! If there's an issue, we can generally help.
• For downloaded items, please provide proof that the software has been uninstalled from all your computers and will never be installed or used any more (screenshots will suffice).

If you have purchased your item by credit card the refund is re-credited to the account associated with the credit card used for the order.

If you have purchased your item by PayPal the refund is re-credited to the same PayPal account used to purchase the software.

If you have purchased your item using any other payment method, we will issue the refund by BACS, once approved by our Financial Department.

For any questions concerning this policy, please contact us at support.

Resellers can purchase an Log File Analyser licence online on behalf of a client. Please be aware that licence usernames are automatically generated from the first and last names used in the Billing Address entered during checkout. If you require a custom username, then please request a PayPal invoice in advance.

For resellers who are unable to purchase online with PayPal or a credit card and encumber us with admin such as vendor forms, we reserve the right to charge an administration fee of £50.

The software needs to be downloaded from our website, the licence key is delivered electronically by email.

There is no part number or SKU.

We do not offer discounted rates for resellers. The price is GBP at £99 per year, per user.

On our contact page.

Licensing details can be found here.

Screaming Frog is a UK based company, so this is not applicable.

No, we only sell in GBP.

In short: The Log File Analyser stores all data in a database on disk. The faster the disk is, the faster you will be able to import and analyse logs.

Memory: All data is stored in a database, so only 1GB is required.

CPU: Any modern CPU.

Hard Disk: Ideally an SSD. This will improve database response times which will make the importing and analysing of data faster. Space taken on disk is roughly 2.5 times that of an uncompressed log file. So if a file is 1GB in size, it will take up roughly 2.5GB of disk space.

This varies depending on operating system, please replace USERNAME with the correct username for the operating system:

• ​Windows: C:\Users\USERNAME\.ScreamingFrogLogfileAnalyser\projects\
• ​Mac: /Users/USERNAME/.ScreamingFrogLogfileAnalyser/projects/
• Linux: /home/USERNAME/.ScreamingFrogLogfileAnalyser/projects/

This can also be viewed and configured within the Log File Analyser, by navigating to 'Configuration > Workspace'. You can select 'Browse' and choose an alternative location for Log File Analyser projects.

The Log File Analyser needs to know what time a log event was generated. It can find this by reading separate date time fields, or a single timestamp entry in the log. Currently supported formats are listed below.

Date formats

• 25/10/2023
• 10/25/2023
• 2023-10-25

Time formats

• 07:02

Timestamp formats

• 25/Oct/2023:12:30:30 +0000
• 2023-10-25T12:30:30+00:00
• 2023-10-25 12:30:30 UTC
• 2023-10-25 12:30:30 +0000
• Oct 25, 2023 @ 12:30:30.000
• 25/Oct/2023:12:30:30.00
• 2023-10-25T12:30:00.000000Z
• 1698233445
• 1698233445000
• 1698233445000000000
• 1698233445.000

TL;DR: The Log File Analyser will import evergreen Googlebot and Bingbot by default.

Google announced they were updating the Googlebot user agent string to reflect the browser version they are using for rendering from December 2019.

Google's Old User-Agents

Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Desktop: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Desktop: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36

Google's New User-Agents

Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Desktop: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Desktop: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36

The key attribute that now changes is 'Chrome/W.X.Y.Z', which reflects the browser version used by Googlebot for rendering. For example, 'Chrome/80.0.3987.92'.

Importing Evergreen Googlebot In The Log File Analyser

We have updated the Log File Analyser to import evergreen changing user-agents out of the box from version 4.0 with improved wildcard matching.

This means no configuration is required and we recommend updating the Log File Analyser if you're using an older version of the software.

As always, we highly recommend using the verify bots feature to ensure user-agents strings are not being spoofed.

The most likely reason is the file you are importing is not a log file, or more specifically, an access log file. Please read our introduction to log files guide that gives examples of the supported log file formats so you can compare with your own.

Common formats imported that are not log files are:

• CSV files
• JSON files
• Apache error logs

The Log File Analyser supports log files in W3C and Apache format. These logs are produced by the 3 most popular web servers: Apache, IIS and Nginx. There is no need to specify what format your logs are in, the Log File Analyser will auto detect this when you upload them.

If you have any problems importing logs, then please contact our support team.

Please see our user guide section on importing log files that lists all our supported formats.

Please contact support if your log cannot be read.

The Apache Common Log Format (CLF) is usually the default format configured for many different web servers.

Unfortunately this format does not provide a User Agent field which is an integral part of the analysis provided by the Log File Analyser.

You will have to reconfigure or ask the administrator of the web server to reconfigure the log format string to provide the user agent field.

Here is the typical NCSA log format line required:

"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

The Log File Analyser uses a database to store the data. So the volume of logs that can be stored will be limited by the size of your hard drive. Performance will also be limited to how fast your disk is - an SSD will give you greater performance than a spinning disk.

The size of your site is not a relevant factor, it's how much data you have in your log files. A small but busy site could have a lot more log events than a large site that doesn't get much traffic.

We often import several Gigabytes worth of uncompressed log files without issue. The larger the time frame you request data for, the more events that have to be considered, and the slower the response will be.

If you have an access log that contains entries for both http and https traffic, but the lines don't include the protocol, the Log File Analyser has no way of knowing which lines are for http and which are for https. The same is true if you have an access log that contains entries for multiple domains. Theses are both very much log file configuration issues. If no protocol and/or domain is specified, you'll be asked for it.

To include the protocol and domain for Apache style logs, the request line should be an absolute url. So rather than:

[01/Jan/2017:00:00:01 +0000] "GET /seo.html HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

The full protocol and domain will be on each line:

[01/Jan/2017:00:00:01 +0000] "GET http://example.com/seo.html HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

This way the Log File Analyser can clearly see the protocol and domain for each line.

For W3C logs, you'll need to include the cs-protocol (Advanced Logging needed for IIS) field along with either cs-host or x-host-header.

The Log File Analyser discards all events (log lines) where the User Agent doesn't match those configured for the project. By default the Log File Analyser only imports events for a 'Bot' User Agents. If you receive this message it's likely you have imported a log that contains events for other User Agents (Chrome/Firefox etc) but not for any Bot User Agents (Googlebot/Bingbot etc). To disable all User Agent filtering create a new project and untick the "Filter User Agents" option when creating a new project (under the 'User Agents' tab). This option can only be set when creating a project.

By default the Log File Analyser only stores events from known search bots. This minimises the number of events imported and speeds up analysis. If you would like to see all events, untick the "Filter User Agents" option when creating a new project (under the 'User Agents' tab). This option can only be set when creating a project.

• Google Search Console (GSC) numbers are for all Google User Agents, the numbers shown in the Log File Analyser are on a per User Agent basis, and do not include all of Google's.
• GSC figures are calculated using Pacific Daylight Time (PDT), however the Log File Analyser uses UTC by default. UTC is 7 hours ahead of PDT. You can adjust the UTC offset to match PDT by going to 'Project > Settings' and choosing UTC-07:00 from the dropdown.
• 'Pages crawled per day' in GSC is all requests. If the same URL is requested 10 times, this is regarded as 10 pages.
• Your log might be from a web server that sits behind a cache, so you're not seeing all the requests.
• Your site might be load balanced across multiple servers and you've not imported logs from all the servers.

The Log File Analyser requires minimal information to import logs: Timestamp, URL, Response Code and User Agent. All other data is optional. If you are seeing "Average Response Time" of 0 then this will be because the log files imported did not include this information. To verify this you can look at the log files you imported to see what was included. If you are unfamiliar, please check our post An SEOs Guide To Log Files.

To add the missing data to your log files you will have to update you log configuration settings, which your web server administrator will be familiar with. Log formats supported by the Log File Analyser are detailed in each of the specific log file formats posts linked to from An SEOs Guide To Log Files.

The Screaming Frog Log File Analyser is obviously analysing data over a period of time, directly from the server logs. So a URL might have been broken historically and subsequently been fixed, and hence why it has different or 'inconsistent' responses.

Another important thing to remember is, if you have uploaded a log file with relative URLs and had to provide the site URL on import, the non www. and www. version of URLs will be aggregated. So events to https://screamingfrog.co.uk and https://www.screamingfrog.co.uk would be aggregated to https://www.screamingfrog.co.uk if that's the address provided upon import. The non www. version might be set-up to 301 redirect to the www. version, and hence, half the responses redirect, while the other half provide 200 responses. This is why we generally recommend logs are configured with absolute URLs.

However, you may also find that a server is delivering inconsistent responses when under load.

The Log File Analyser reads response codes from log files and displays them. If these are different to what you see by checking using a web browser or the SEO Spider there are a few things to check.

• Some pages respond differently based on user agent, cookies etc. If you look through the history of response codes for the URL, by clicking the Events tab in the lower window pane, do the response codes vary?
• Are the logs you've imported for the correct domain? If you had to provide a protocol and domain during import, there are a couple of potential issues. Perhaps the logs are for both the http and https versions of the site or maybe the logs are for multiple domains.

If the Log File Analyser user interface isn't rendering for you then the chances are you've run into this Java bug. On our experience this seems to be an issue with Intel HD 5xx series graphics cards. We've had less of these recently, so it might be a drive update will help resolve this issue.

If not, please close the SEO Spider, then open up the following file in a text editor:

C:\Program Files (x86)\Screaming Frog Log File Analyser\ScreamingFrogLogFileAnalyser.l4j.ini

then add the following under the -Xmx line:

-Dprism.order=sw

(You may have a permission issue here, so copying your desktop, editing, then copying back may be easier).

No when you start the Log File Analyser the user interface should render correctly.

Try running the file as administrator by right clicking the installer and choosing “Run as administrator”. Alternatively log in to an administrator account. You may need to request assistance from your IT department depending on your company setup.

When you install the Log File Analyser on Windows 10, you may receive a 'protected your PC' SmartScreen - To install, you have to click the 'More Info' link underneath and then 'Run Anyway' - The reason for this is the certificate is new and it takes time for this message to disappear, despite being registered Windows developers.

As of version 5 we now ship two versions of the Log File Analyser; one for Apple Silicon processors that were introduced in late 2020, and one for the previous generation of machines that used Intel processors. You need to select the correct download based on the type of processor your machine has.

You can identify what processor your machine has by clicking the Apple logo in the top left of your screen and selecting "About This Mac". You can now see what machine you have by looking at the value for Processor/Chip. If this says "Apple M1", "Apple M2", or "Apple M3", you have an Apple Silicon machine. If it says "Intel" anywhere, then you have an Intel machine.

If you have an Apple Silicon machine, please download the version labelled "macOS (Apple Silicon)".

If you have an Intel machine you will most likely need the version labelled "macOS (Intel) version 11 or later". If you are running macOS version 10.14 "Mojave" or 10.15 "Catalina" you will not be able to use version 19.3 or later unless you update your operating system. The latest version you'll be able to use is 5.3 which can be downloaded by choosing "macOS (Intel) for 10.14 or 10.15".

You will not be able to update your operating systems if your machine is too old, in which case you will have to remain on Spider 19.2 until you update your machine.

You can get the debug logs from the Log File Analyser by going to Help->Debug->Save Logs. This will allow you to save a zip file containing the logs we need to help diagnose any issues.

If for some reason you are unable to start the Log File Analyser locate and attach the trace.txt file. This is found in a different location depending on your OS. Please substitute USERNAME for your own username.

Windows:
C:\Users\USERNAME\.ScreamingFrogLogfileAnalyser\trace.txt

OS X & Linux:
~/.ScreamingFrogLogfileAnalyser/trace.txt

Please follow the steps on the Log File Analyser support page so we can help you as quickly as possible. Please note, we only offer full support for premium users of the tool although we will generally try and fix any issues.

Feedback is welcome, please just follow the steps on the support page to submit feedback. Please note we will try to read all messages but might not be able to reply to all of them. We will update this FAQ as we receive additional questions and feedback.