I was able to resolve the issue on the server side. It turned out that our apache mod_ssl was configured to allow only TLSv1.2 and that the cipher suite was also restrictive. Using https://www.ssllabs.com/ssltest was helpful in analyzing.
Conclusion - we updated the allowed SSL protocols to include TLS v1, 1.1 and 1.2 and added the following cipher suites in this order:
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5
We're now able to login to our app using TLS on Android OS's below 5.0.