Fixing your dedicated IP sending reputation with outlook/hotmail

If you have a mail server with your own dedicated address that you host your own web sites on and send your own emails then you could have your emails rejected by Microsoft properties such as hotmail and outlook.com for no reason other than the IP address space you are in (e.g. the /24 ) has had spammers even though your specific IP address doesn’t SPAM.

You’ll get a rejected message such as,

550 5.7.1 Unfortunately, messages from [x.x.x.x] weren't sent. Please contact your Internet service provider since part of their network is on our block list (AS3140). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com]

All is not lost as you can have the IP mitigated and removed from the block. To do this you’ll have to do the following,

  1. Make sure that your SPF records are acceptable. Without valid SPF records then you’ll find that your emails will be marked as SPAM by servers such as gmail anyway.
  2. Register your IP addresses in the Smart Network Data Services program (SNDS). From this you can monitor the ‘health’ and reputation of your registered IPs by providing data about traffic such as mail volume and complaint rates seen originating from your IPs.  It is in the tools of the SNDS that you will see which of your IP addresses are blocked. To register, visit http://postmaster.live.com/snds/
  3. Then register for the Junk Email Reporting program (JMRP) so that when an Outlook.com user marks an email as “junk”, you will get a copy of the mail. This will allow you to see what you are sending is being marked as junk and to track down email traffic you did not send. To join, please visit http://support.msn.com/eform.aspx?productKey=edfsjmrpp&page=support_home_options_form_byemail&ct=eformts
  4. Remember to CONFIRM the SNDS/JMRP agreement that is sent to your registered email, This is an electronic signature using the Docusign system.
  5. Once you have done this then leave this for 48 hours and confirm that you are not getting any SPAM reports. It is unlikely anyway if it is just an IP that is blocked but the SNDS/JMRP changes take 24-48 hours to propagate.
  6. Now you can request a mitigation of your IP block using the following link, https://support.microsoft.com/en-us/getsupport?oaspworkflow=start_1.0.0.0&wfname=capsub&productkey=edfsmsbl3Fill out the details demanded by the form and submit. Fairly quickly within an hour you should get a number of emails posted to your contact email and hopefully,  if your IP address was just accidentally been discriminated against because of the address block, get a reply that your IP address has been conditionally unblocked (“These IP(s) have been unblocked, but may be subject to low daily email limits until they have established a good reputation.“) . It’ll still show up blocked in SNDS for another day as the blocked list is only updated daily. After a day then your IP should disappear from the SNDS system.

    If you are bulk mailing people then it is simpler and far easier to do that via a bulk emailer such as Mailchimp or  Mailerlite and to not use your own server. Leave your server for your own individual emails.

Fixing the Western Digital excessive load/unload problem with idle3-tools

I have an always-on machine (self made Ubuntu 16.04 based using retail parts) that is lightly loaded in CPU but is continually writing to the hard disks. I recently added a Western Digital 2TB drive ( WDC WD20EZRZ-00Z5HB0 ) about 2 weeks ago (368 hours) and noticed that the Load/Unload cycles was over 30,000.
The other two drives, both WD Blue 1TB ( WDC WD10EZEX-08M2NA0 ) showed up with less than 100 on 15,000 hours so something was very wrong with this new drive in my system.
The fix is to adjust the wdidle3 setting from its default of 80, which means 8.0 seconds, to a bigger number such as 129, which means 30 seconds.

The number on newer drives is not just divided by 10 but is staggered scale so 1-128 is divided by 10 but 129-255 is in 30 seconds increments (129 = 30sec, 130 = 60sec and so on) for newer drives but it is just divided by 10 for older drives. I do not know what is deemed new or old manufacturing date for WD drives.

You can read and set the wdidle3 parameter using hdparm -J option but it has a very cautious warning message as the option is experimental.

You can read the S.M.A.R.T value for the Load/Unload using smartctl from the command line, (or GSmartControl using a GUI) and the ID is 193 thus,

sudo smartctl -A /dev/sdc | grep "^193"

I get…

193 Load_Cycle_Count 0x0032 190 190 000 Old_age Always - 31283

where the 31283 is my current, but now stable, excessive count.

To set the parameter I downloaded and used the idle3-tools from http://idle3-tools.sourceforge.net/ This is a small program that you just untar and make or you can install it with,

sudo apt-get install idle3-tools

I set my drive with…

sudo idle3ctl -s 129 /dev/sdc

…which for new drives means 30 seconds ( and -s 130 would mean 60 seconds and so on) . After you set the value in the drive you must turn off the power to your computer (drive) as the value is read at power up. So do a shutdown and then power off.

I have no idea why Western Digital do what they do but it is not that friendly unless you delve deep. Reading the WD support forums the problem is from many years ago and is still taking place today. I expect to be able to install a disk and not have it artificially age itself but adapt to the load. Equally I have no idea yet if the 30 second value I have set is a better compromise. It has stopped the value increasing and if my programs stall then the disk will unload reasonably early.

FreeSWITCH bootstrap libtool not found – missing libtool-bin

Was rebuilding FreeSWITCH source and had upgraded my test machine Ubuntu from 14.10 to version 15.04 so I did a make uninstall on FreeSWITCH and reran the bootstrap.sh but it came up with the error of libtool not found.

Libtool (2.4.2) was installed but what I found is that I had to add libtool-bin with,

sudo apt-get install libtool-bin

Then bootstrap.sh worked fine.

Opera slow Flash due to multiple plugin locations enabled

I use Opera as one of a number of browsers for testing purposes. Noticed that Flash was jumpy (Opera 12.16 with flash 11.2 r202 on Ubuntu 14.04 64 bit). It took a while to find the problem but I believe that it was due to multiple flash plugins enabled. To see what plugins are enabled go to,

Opera -> Page -> Developer Tools -> Plug-ins

(or the shortcut URL of opera:plugins )

If you see multiple locations of the Shockwave flash enabled then disable all except one e.g. leave the one located at,  /usr/lib/mozilla/plugins/libflashplayer.so enabled.

Restart Opera and hopefully this may clear your problem.

memtest86+ cannot load a ramdisk with an old kernel image

This error happens when you use UNetbootin to create an Ubuntu disk and it incorrectly adds a ramdisk to the memtest86+ boot option.

Until UNetbootin fix their code then cursor down to the “Test memory” option and hit tab and then at the boot options remove the “initrd=/ubninit” so that the command line is now just…

/install/mt86plus

and then hit enter and Memtest86+ will now run as expected.

My Ubuntu 14.04 currently has UNetbootin 585-2ubuntu1 and this quirk will possibly be fixed in newer releases but sometimes all you have lying around is an emergency install USB/disk so always good to know how to get around  a problem rather than downloading new code.

tentacle server not logging agent data after Pandora FMS upgrade

If you are using the password option for the tentacle protocol then you must also add the password to the TENTACLE_EXT_OPTS in the /etc/init.d/tentacle_serverd  file e.g.

TENTACLE_EXT_OPTS=”-x YOURAGENTPASSWORD  …

If you do not do this then if you are using the agent password option then no agent data will be logged.

This is not a very secure way of data logging.

 

 

Removing Delta Search Open Tab in Internet Explorer and Firefox

Delta Search is ad-ware that hooks into your browser experience. It appears to leave behind a few bits when it is removed. The last bit to remove that I found is that Delta Search is launched when you open a new tab in Internet Explorer or Firefox.

For Windows Internet Explorer: in Windows use regedit then you can search for delta-search
The change is,

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\AboutURLs\Tabs

which should be as a default,

res://ieframe.dll/tabswelcome.htm

For Firefox: in Firefox go to the special URL about:config and find the parameter,

browser.newtab.url      

which should be

about:newtab

Opera Start tab doesn’t work in the same way so it doesn’t have this issue.

vTiger 5.4.0 enable backup quirk

Note that from vTiger 6.0 onwards i.e. 6.5 and 7+ there is no default backup module. You’ll either have to buy a commercial module or roll your own script. The guide below will be obsolete.

I use the vTiger CRM product and it has a unusual quirk with enabling backups. Whilst the user interface (CRM settings -> Backup server) has check boxes to enable local and FTP backup the script actually tries to alter the following file, /user_privileges/enable_backup.php and in that set the two flags to a value of either true or false,

$enable_local_backup = 'true';

$enable_ftp_backup = 'false';

As the script has no permissions to do that then it gets a fopen() permission error (Warning: fopen(/*****/user_privileges/enable_backup.php): failed to open stream: Permission denied in /****/modules/Settings/SaveEnableBackup.php on line 43) and so when the ajax refreshes the screen it looks as if nothing has been done.

Without messing with your directory permissions then you can edit the /user_privileges/enable_backup.php manually.

Even if you manage to get the directory details into the local backup it will not work unless that enable local backup flag is set to true. It will say that it has done a backup but it will not save any file.

Mailfilter fails to POP timestamp in message-ID invalid

Mailfilter POP timestamp in message-ID invalid and potentially also causes mailfilter to stall at 100% CPU.

I have a fetchmail daemon that call mailfilter as a postconnect (defined in the .fetchmailrc file). I got the following error message,
mailfilter: Examining 297 message(s).

mailfilter: Error: POP timestamp in message-ID invalid.
mailfilter: Error: Parsing the header of message 292 failed.
mailfilter: Error: Scanning of mail account failed.
mailfilter: Error: Skipping account xxxxxx@example.com@mail.example.com due to earlier errors.

and I noticed that the mailfilter process was running at 100% CPU though that may be unrelated.

I found that mailfilter 0.8.3 has a new option of -i to ignore POP timestamp. This is probably what I want to make this more stable.

As I’m adding this to a Parallels based server it is unlikely to have this version of mailfilter as this package has very little development activity as it is a stable application so I had to build from source.

My server didn’t have svn so I browsed the Sourceforge svn for mailfilter on my local PC and at the bottom there is a linkfor “Download GNU tarball” I copied that link and then pasted into my console and used wget to get this latest tarball from Sourceforge. It has a odd name so moved that to a tar.gz file name e.g. mv index.html\?view\=tar mailfilter.0.8.3.tar.gz and then ran tar xvfz mailfilter.0.8.3.tar.gz

Perquisite packages for building,

  • g++
  • bison
  • flex
  • libssl-dev

There may be more but that is the ones I needed to add to my server.

Making this you cd to the mailfilter directory and then run

./autogen.sh
make
sudo make install

If that works then this’ll probably install the mailfilter to /usr/local/bin/mailfilter so now in the .fetchmailrc change the postconnect line to have,

postconnect ''/usr/local/bin/mailfilter -i ''

use double quotes if you pass the new -i option to ignore timestamps. The -i is a new feature in Mailfilter 0.8.3 (not in 0.8.2).

Killall the mailfilter and fetchmail and then re-launch your fetchmail daemon.

Errors

Missing g++

If you see,

checking whether the C++ compiler works... no
configure: error: in `/root/sources/mailfilter':
configure: error: C++ compiler cannot create executables
See `config.log' for more details.

Then check the log file and look for g++ line e.g.

configure:2879: checking for g++
configure:2909: result: no

Do a g++ and if it comes back with -bash: g++: command not found then install the g++ package and then it will work. Re-run the ./autogen.sh and then make

Missing bison

If you get the make fail and you can see /bin/sh: yacc: command not found then you need a YACC of some kind. I installed bison re-run the ./autogen.sh and then make

Missing flex

If you get an error in the make e.g. it crashes out with g++: rcfile.cc: No such file or directory

g++: no input files
make[2]: *** [rcfile.o] Error 1

then check back and see if you see error: FlexLexer.h: No such file or directory . If so then check flex is installed. Install and then re-run ./autogen.sh and then make

Missing openssl header files
If you see openssl/ssl.h: No such file or directory and similar openssl/rand.h: No such file or directory then you need to install libssl-dev. Install that package and then re-run ./autogen.sh and then make.

After a month and a few weeks of use it has been stable. The erroneous timestamps are also suspected to stall Outlook 2003. I pick up emails in parallel to my customer to provide emergency support when they are on holiday. As Microsoft support for both XP and Office 2003 is finishing in April 2014 I plan to migrate the customer to a newer OS and Office version.

Enabling anchor for Disqus comments on WordPress site

When you add Disqus to your site e.g. as I did with http://www.lincolnphipps.org then make sure that the following Disqus plugin advanced option is set,

Comment Counts   [ * ] Output JavaScript in footer

If it is not checked then when you click the post and then click the (theme dependant) link that is usually at the top of a post to jump to the comments at the bottom of the post, it does nothing. The default WordPress anchor tag is #comments but with Disqus it is #disqus_thread

When this disqus plugin advanced option is checked then the correct anchor is used and you will jump to the Disqus comments as expected.

 

vTiger CRM missing language file causes Sorry! Attempt to access restricted file.

When you add a new user (in vTiger CRM v5.4.0) then it is possible that you can select a language setting e.g. “English GB” that has a non-existent translation file. If you do this then that user will get the error message,

Sorry! Attempt to access restricted file.

on a blank screen after they log on. They will stay in this state forever until the logon cookie is removed or the language file is created. The language files exist under the path,

include/language

To fix this you must either

  • upload the correct language file or
  • use another browser and go in as a working user e.g. admin and reset the language to a known working language e.g. English US or
  • clone the include/language/en_us.php to the correct file name e.g. en_gb.php. To identify what file is needed you can add a print_r($filepath); to the include/utils/CommonUtils.php file function checkFileAccessForInclusion() around line 2817 onwards but before the die(). You can’t keep that debug code in place as the ajax javascript breaks but it will tell you what language filename it is after.

That should solve this first-day-of-use problem.

Typical DDI trunking connection problems

Once you have your FreeSWITCH setup then adding new trunks should work fine but you can get some first-day-of-use problems that slow you down.As I find them I will update this post.

Calling PSTN number of DDI trunk, rings then silence.
This is probably due to the trunk on FreeSWITCH not having the  “Register” checkbox set (true). If using bluebox then in the Connectivity->Trunk Manager for the trunk check the Register checkbox. In XML this is in the conf/sip_profiles/ in your .XML file for the profile of the interface you need to verify that register is true,

       <gateway name="trunk_x">
        <param name="realm" value="sip.example.com"/>
        <param name="username" value="xxxxxxxx"/>
        <param name="password" value="xxxxxxx"/>
        <param name="register" value="true"/>
        <param name="extension" value="auto_to_user"/>
        <param name="auto_to_user" value="true"/>
      </gateway>

Calling PSTN number of DDI trunk, rings then very quickly goes to disconnect.
This is due to the inbound ACL not being set to “Trunks (Auto)” on the Sip interface (NAT).

 

Always getting number unavailable with Cisco dial plan

Dial plans are fairly easy to read but there is one little gotcha if you enter in one of these incorrectly: you will always get number unavailable.

You need to make sure that the dial plan you have entered in is valid i.e. it uses the | pipebar for delimiters and has no spurious close brackets ‘)’

The web interlace of a Cisco IP phone does not validate your input.

Enabling WordPress comments

I just enabled WordPress comments on my own personal pages. For the moment I’ve enabled the SI Captcha plugin and to remove the default website link in the comment form I added the urlfilter.php file trick that I found here.

You do not need to be registered to logon but you need a name and email but all comments are moderated.

To retrofit enabling comments to older posts then you go into the All Posts dashboard option, check the box next to the title to select all the posts and then in the Bulk Actions you select Edit and then Apply. This will open a lot of selection and options. Pick the Comments option of Allow and then do Update.

Chunky text rendering in Adobe Acrobat Reader

Yuk yuk renderingI was viewing some PDF files on a PC and came across  a rather odd result. The PDFs were displaying text very chunky. An example is to the right.

The document prints perfectly; it is only the screen display that is chunky.

To fix this go into Adobe Acrobat Reader menu option Edit -> Preferences then in that in Page Display under “Rendering” make sure that the “Smooth Text” is set to something other than None. This fixes this issue.