I ran into an issue where mail from my server/domain ended up in Gmail users’ spam folders and so endeavored to resolve it. I didn’t find anything in my online searching where someone was seeing exactly what I was seeing.

This link was helpful for general information around SPF: http://www.openspf.org/FAQ/Common_mistakes

For purposes of this documentation, configuration/values used:

Server IP : 192.168.1.1
Server hostname : host.domain.com (yes, different than example.com domain)
SMTP HELO (exim4) name : mail.example.com

Personal address: user@example.com
Gmail address: user@gmail.com

As for the symptoms, most importantly, email was ending up in Gmail users’ spam folders. In viewing the email header, I could see the reason was a softfail:

Received: ⁨from mail.example.com (host.domain.com. [2001:61f5:41:82c::235]) by mx.google.com with ESMTPS id u26si4316688wrd.422.2017.10.26.12.05.13 for <user@gmail.com> (version=TLS1_2 cipher=AES128-SHA bits=128/128); Thu, 26 Oct 2017 12:05:14 -0700 (PDT)⁩
Authentication-Results: ⁨mx.google.com; spf=softfail (google.com: domain of transitioning user@example.com does not designate 2001:61f5:41:82c::235 as permitted sender) smtp.mailfrom=user@example.com

Now I never did find out (neither from hosting provider nor in my online searching) why Google is printing what appears to be an IPv6 (hexadecimal) address in the response (generally, you should see the IPv4 address of the server instead) log it adds to the header, but I did get the gist that the host was not explicitly authorized to send mail on behalf of example.com.

I’m not going to go into detail about SPF usage (see link at beginning of post), but when sending email from a domain, a SPF DNS entry is required for that domain so that receiving servers can validate email by linking the sending machine to the domain (the SPF entry defines which machines can send mail on behalf of the domain). A single entry may contain multiple rules, separated by spaces, which are read sequentially until one is satisfied. If none are satisfied, the “*all” rule at the end directs the calling server how/if to fail the email message.

At the time I was encountering this issue, the SPF entry for my domain looked like:

example.com IN TXT "v=spf1 ip4:192.168.1.1 a:mail.example.com ~all”

My impression was that specifying the server IP address here (from which email will be sent) would satisfy ISP checks: all emails sent from 192.168.1.1 would be caught and validated by the rule “ip4:192.168.1.1,” and both mail.example.com (HELO name) and host.domain.com (host name) resolve to 192.168.1.1. Evidently, this was not the case. I viewed the addition of “a:mail.example.com,” seen above, as extraneous but worth a shot (read: desperation). Still, no dice. Note: the ‘~’ part of ‘~all’ directs a server to softfail a message if no rules in the entry are satisfied, as opposed to ‘-all’, which directs a server to hardfail.

I initially suspected Google was performing a HELO/EHLO (hereon just “HELO”) check that was failing due to no SPF entry for mail.example.com. Indeed, to satisfy servers which employ 100% HELO checks (or those scenarios where Mail from is empty in the message), a separate SPF entry is required for the HELO name itself (this is a best practice, though I’m not sure how often this check is employed). And so I added an appropriate DNS entry:

mail.example.us IN TXT "v=spf1 a -all"

In other words, explicitly allow email from this HELO FQDN (via the “a” rule), mail.example.com. This resulted in no change; I was still encountering the softfail.

Next I homed on the fact that, in addition to reporting the HELO name, Google was printing my actual hostname in the email header:

Received: ⁨from mail.example.com (host.domain.com. [2001:61f5:41:82c::235]) by mx.google.com with ESMTPS id u26si4316688wrd.422.2017.10.26.12.05.13 for <user@gmail.com> (version=TLS1_2 cipher=AES128-SHA bits=128/128); Thu, 26 Oct 2017 12:05:14 -0700 (PDT)⁩

See that host.domain.com? That’s the hostname of my server. More specifically, it’s the name associated (via rDNS) with IP 192.168.1.1. Google must be performing a reverse DNS lookup to retrieve that hostname. I wondered if it was then performing an SPF check based on that name, so I added a discrete rule for it in my SPF entry:

example.com IN TXT "v=spf1 ip4:192.168.1.1 a:host.domain.com ~all”

This fixed it:

Received: ⁨from mail.example.us (host.domain.com. [2001:61f5:41:82c::235]) by mx.google.com with ESMTPS id p19si797419wrf.42.2017.11.01.08.37.26 for <user@gmail.com> (version=TLS1_2 cipher=AES128-SHA bits=128/128); Wed, 01 Nov 2017 08:37:26 -0700 (PDT)⁩
Authentication-Results: ⁨mx.google.com; spf=pass (google.com: domain of user@example.com designates 2001:61f5:41:82c::235 as permitted sender) smtp.mailfrom=user@example.com⁩

Success! Still not sure what’s up with the IPv6 address, but success, nonetheless.

Upon further investigation (i.e., empirical testing), it appears that Google was not using a HELO check at all in this scenario, which I suppose isn’t to say it never does or never will.

So for Gmail, it looks like the most important thing is for that domain name resolving via rDNS (in my case, host.domain.com) to be present as explicitly ‘allowed’ in the SPF entry instead of relying on the IPv4 rule alone. Whether this would be required if the hostname and HELO name are the same, I don’t know. And whether that IPv6 address getting returned instead of IPv4 has anything to do with it, also not sure, but I’m eyeing that warily and have reached out to my hosting provider.

In summary:
– To satisfy Google (and other ISPs performing check via rDNS) spam check, add rule for server hostname (more specifically, the FQDN returned from rDNS of server IP) as explicitly allowable sender in SPF record for any and all domains from which mail will be sent.
– Additional best practice (not sure how often/ever this is implemented by an email server): and add new SPF entry for HELO name (e.g., “v=spf1 a -all”).

I required the following:

• System user that could upload content to a directory in root web directory (default root: /var/www/html)
• Limit user from interactive SSH
• Limit user from other areas of OS

Specifically, I am working within the AWS distribution on a hosted EC2 instance.

I found posts online that accomplished part of what I needed. But my steps to achieving this were:

1. Create the user. In my case, user webpub. This creates an entry in /etc/passwd as well as a home directory under /home: 

   sudo useradd webpub

2. These next few steps I found here. Create a ‘jail’ directory that we will constrain the user. I created it in /var.

   sudo mkdir /var/jail
   

3. An important note is that the jail directory and all directories beneath it must be owned by user root in order for the Chroot declaration to work. If you get setup and notice that you are correctly authenticating but then the connection immediately drops, this could be your problem. Now create a sub-directory that will serve as the access point for the user to the web content:

   sudo mkdir /var/jail/www

4. The directory created above can also be owned by root. Create a sub-directory under web content root that we will restrict this user to. In this case, the same name as the user:

   sudo mkdir /var/www/html/webpub

5. The directory created above can also be owned by root. Now create the link between the jail and the content directory by binding the two:

   sudo mount -o bind /var/www/html/webpub /var/jail/www

6. In /etc/passwd, update the user webpub‘s home directory (where they will land upon logging in) to /var/jail/www.
7. Update /etc/ssh/sshd_config to jail the user upon logging in. Start by commenting the line Subsystem sftp /usr/libexec/openssh/sftp-server and then adding configuration for the internal-sftp sub-system. When done, it will look like (commented line and all): 

   #Subsystem sftp /usr/libexec/openssh/sftp-server
   Subsystem sftp internal-sftp
   Match User webpub
           ChrootDirectory /var/jail
           ForceCommand internal-sftp
           X11Forwarding no
           AllowTcpForwarding no

8. The ChrootDirectory jails the user while ForceCommand internal-sftp lists the user to only being able to login via SFTP. Now restart openssh:

   sudo /etc/init.d/sshd restart

9. In my setup, I have password authentication disabled, so the last step is create a private/public key pair and install client/server side. Remember that authorized_keys (and its parent directory .ssh) must reside in the home directory for webpub, which we set earlier as /var/jail/www. Since that directory is bound to /var/www/html/webpub, though, these artifacts reside in the latter directory.

I did a fresh install of Ubuntu 14.10 today with Cinnamon as a desktop and am pleased with the interface.

I noticed something when scanning some documents in lineart mode, though: the resulting images had a ton of noise, noise that I did not see in scans prior to my upgrade. After snooping around the various options in the gscan2pdf application, I stumbled upon this one which, when toggled, causes the noise to disappear: Disable dynamic lineart. After checking that box, my scans seem to be noise free.

I seem to occasionally find myself sparring with Ubuntu on getting VNC server configured correctly. I use the default (RealVNC) vncserver package included with Ubuntu. There are two main points of configuration I seem to rediscover each time:

Make sure vncserver is looking in the right place for configuration

For my setup, the “right” place means ~/.vnc/xstartup. I’ve seen vncserver look to /etc/X11/Xsession by default. To change this, make sure the following line is in /etc/vnc.conf:

$vncStartup = "~/.vnc/xstartup";

Now you need to make sure the configuration exists in your user home directory. Following is the contents of ~/.vnc/xstartup (note: there are several commented lines I leave there for posterity:

#!/bin/sh
# Uncomment the following two lines for normal desktop:
unset SESSION_MANAGER
#exec /etc/X11/xinit/xinitrc
#gnome-session --session=gnome-classic &
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
#x-terminal-emulator -geometry 1280x1024+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &
startxfce4 &


Then I just have a simple script in my home directory to start a vnc session, looks something like this:

#!/bin/bash
vncserver :1 -geometry 1440x900 -depth 16 -name my_lil_desktop

EDIT (2015-01-23) : Today I installed version 14.10 of Ubuntu. By default, it uses the ‘vino’ application for remote desktop tasks. It did not work out of the box for me, and this seemed to be the case for many, based on what I read online. At any rate, I fell back to the configuration described in this post but had to install vnc4server first, as it was not installed along with 14.10. I initially tried tightvncserver but it did not work with this configuration; vnc4server, however, did.

I discovered today that Samba appears to be uncooperative when trying to connect from a single machine to two or more of its shares using different user names. This is a probably a pretty esoteric problem, I realize, but came into play when I was troubleshooting another user’s account without disconnecting shares mounted under my own user name first. The client machine was running Windows XP, the server Ubuntu 7.10 (Gutsy Gibbon).

Ubuntu has been my Linux of choice for sometime now. Currently using Ubuntu 7.10 (Gutsy) with default VNC package. Along the way I have found some issues.

Keyboard Malfunction

I recently ran into a problem while upgrading distributions. In short, the keyboard mapping was completely messed up when connected over VNC. Typing characters over VNC returned seemingly random characters on the other end. The problem has been discussed and a work-around solution proposed here:

https://bugs.launchpad.net/ubuntu/feisty/+source/vino/+bug/112955

Two posts by “wjs” on that page got me up and running (I am using a U.S. keyboard). The correct characters are now carried over VNC. Evidently, this bug is still not fixed, and has been experienced with multiple Ubuntu distributions. Check out the link above for more information.

Disappearing Desktop

I have also found an issue related to the display of the desktop and menus which seems to affect the operation of the keyboard and mouse, as well. Whenever I would try to connect to a VNC session, the login process would begin, the menu bars at the top and
bottom would flash, but disappear. I was left with only the trademark Ubuntu brown desktop (my current background) and no menus. The mouse and keyboard also seemed inoperable. I began to think something in gnome was broken, but could not track down the problem. In the end, I discovered that specifying a color depth when starting the VNC server corrected the problem. Instead of just

vncserver -geometry 1024x768 :1

which would give me an inoperable desktop, I used

vncserver -geometry 1024x768 -depth 16 :1

and the desktop loaded fine. I have not looked around much online regarding this problem.