Wednesday, October 18, 2023

Bash 101: Relative Pathing

While preparing for an interview in 1992 for a co-operative education program at IBM to support their new RISC SYSTEM/6000 line of servers and it's version of the UNIX operating system, Advanced Interactive eXecutive (AIX), I was introduced to what would become one of my favorite past times, shell scripting.  At that time, the common shells were the Bourne shell (sh), Korn shell (ksh), and C shell (csh).  Later, Linux would introduce what is now the default shell of many UNIX distributions, the Bourne-Again shell (bash).

Though shell scripting is technically not a programming language, everyone that works in the UNIX domain becomes familiar with shell scripting at some level.  Over my 30+ year career in IT, I've easily contributed more than 10,000,000 lines of shell scripting code to the projects that I've worked on.  The purpose of this blog is to pass along some of the gems that I've collected along the way.

When working on projects that include multiple scripts, one of the biggest problems is how do you write all of the scripts such that they will work properly in any directory path. For example, my current project is composed of 84 different scripts.   The answer, is that you define a common path context for all of the scripts in the header of every script in the project.  My standard for this is the following:

#!/bin/bash
cmd=$(basename $0)
curdir=$(dirname $0)
curdir=$(cd ${curdir}; pwd)

Let's unpack this simple set of lines to understand what's going on.

1. Define the shell to use for this script's runtime environment via the shebang directive.

#!/bin/bash

2. Determine the script name.  The $0 variable is the execution path of the script being run.  For example, the execution path could have been to run the script in the current directory with ./myscript.sh.  Or, it could have been called from a directory above with ../myscript.sh.  Or, perhaps it might have been called using the full path of /opt/ods/poc/myscript.sh.  The basename command trims off the path to leave just the script name.  We assign the output of the basename command to the cmd variable so that it can be used as a common reference (${cmd}) throughout the rest of the script.

cmd=$(basename $0)

3. Determine the base directory based of the script's location.  The dirname command trims off the script name leaving just the path to the script.  We temporarily assign the result of the dirname output to the curdir in the first step to determining the current directory.

curdir=$(dirname $0)

4. Lastly, we change directory to the current value of ${curdir} and then use the present working directory (pwd) command to determine the fully qualified path of the directory that we are in.

curdir=$(cd ${curdir}; pwd)

You may be wondering why we didn't just stop at step 3 because technically, that is correctly the current directory.  The reason is because the execution of the script can be called from a variety of ways that are not the full path name.  For example, if I ran the script with ../../../myscript.sh, the first value of curdir would be  "../../../", which is not the fully qualified path.  However, after step 4, the fully qualified path is righly determined to be /opt/ods/poc.

I hope you find this helpful!

Blessings!

Brad



Posted by at No comments:

Tuesday, October 17, 2023

OUD 12cPS4 Use Case: Oracle Database Name Services

Oracle Unified Directory (OUD) is one directory service product included in the Oracle Directory Services Plus (ODS+) suite that is used for a wide variety of use cases requiring LDAPv3 interoperability.  

Oracle Database Name Resolution
One very common use case is Oracle Database name resolution. Oracle name resolution is referred to by several names including "net services", Transparent Network Substrate (TNS), and Oracle Names (ONAMES) depending on the Oracle Database version.

Oracle Database name resolution is to Oracle Databases as Domain Name Services (DNS) is to resolving a fully qualified domain name to an IP address to enable a web browser or ssh client to connect to the associated IP address.  For more information on Oracle Database name resolution (a.k.a. Net Services), see the Oracle Database documentation here: https://docs.oracle.com/en/database/oracle/oracle-database/19/netag/

Basic Workflow
In this use case, Oracle Database clients connect anonymously [default] or via basic authentication to the OUD directory server over either the LDAP [default] or LDAPS protocol.  Once connected to OUD, the database client requests the client connect string for a specific database.  The connect string is returned to the database client and then the database client uses the provided connection string to connect to the Oracle Database.  The following is a sample connect string returned by OUD:

(DESCRIPTION =
  (ADDRESS=(PROTOCOL=TCP)(HOST=node1.example.com)(PORT=1521))
  (CONNECT_DATA=(SERVICE_NAME=node1.example.com))
)

Once the database client has the connect string, it uses the information in the connect string to connect to the Oracle Database.

OUD Setup
The following is the basic workflow for setting up a pair of OUD instances for this use case on the same server:


1. Make the directory where the software will be installed

$ sudo mkdir /opt/oud/bits
$ sudo chown -R opc:opc /opt/oud

2. Download JDK and OUD into /opt/oud/bits
3. Extract the JDK and OUD software and patches

$ tar --directory=/opt/oud -zxf /opt/oud/bits/jdk-8u*-linux-x64.tar.gz
$ mv /opt/oud/jdk1.8.0* /opt/oud/jdk
$ unzip -d /opt/oud/bits/. /opt/ods/poc/bits/V983402-01.zip
$ unzip -d /opt/oud/bits/. /opt/ods/poc/bits/p28186730_139427_Generic.zip
$ unzip -d /opt/oud/bits/. /opt/ods/poc/bits/p35263333_122140_Generic.zip


4. Create input files for OUD installation and Oracle inventory

OUD installation response file (/opt/oud/bits/oud12c-standalone.rsp):

[ENGINE]
Response File Version=1.0.0.0.0
[GENERIC]
DECLINE_AUTO_UPDATES=true
MOS_USERNAME=
MOS_PASSWORD=
AUTO_UPDATES_LOCATION=
SOFTWARE_UPDATES_PROXY_SERVER=
SOFTWARE_UPDATES_PROXY_PORT=
SOFTWARE_UPDATES_PROXY_USER=
SOFTWARE_UPDATES_PROXY_PASSWORD=
ORACLE_HOME=/opt/oud/mw_oud12c
INSTALL_TYPE=Standalone Oracle Unified Directory Server (Managed independently of WebLogic server)


Oracle inventory file (/opt/oud/bits/oraInventory.loc)

inventory_loc=/opt/oud/oraInventory
inst_group=opc


Temporary password file for setup (/opt/oud/bits/.pw)

$ echo 'Oracle123' > /opt/oud/bits/.pw

Java security configuration file (/opt/oud/bits/tns-java.security) that allows anonymous cipher suites, which are required for registering databases with dbca

jdk.disabled.namedCurves = secp112r1, secp112r2, secp128r1, secp128r2, \
    secp160k1, secp160r1, secp160r2, secp192k1, secp192r1, secp224k1, \
    secp224r1, secp256k1, sect113r1, sect113r2, sect131r1, sect131r2, \
    sect163k1, sect163r1, sect163r2, sect193r1, sect193r2, sect233k1, \
    sect233r1, sect239k1, sect283k1, sect283r1, sect409k1, sect409r1, \
    sect571k1, sect571r1, X9.62 c2tnb191v1, X9.62 c2tnb191v2, \
    X9.62 c2tnb191v3, X9.62 c2tnb239v1, X9.62 c2tnb239v2, X9.62 c2tnb239v3, \
    X9.62 c2tnb359v1, X9.62 c2tnb431r1, X9.62 prime192v2, X9.62 prime192v3, \
    X9.62 prime239v1, X9.62 prime239v2, X9.62 prime239v3, brainpoolP256r1, \
    brainpoolP320r1, brainpoolP384r1, brainpoolP512r1

jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
    DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, NULL, \
    include jdk.disabled.namedCurves

jdk.tls.legacyAlgorithms= \
    K_NULL, C_NULL, M_NULL, \
    DH_anon, ECDH_anon, \
    RC4_128, RC4_40, DES_CBC, DES40_CBC, \
    3DES_EDE_CBC


5. Install the OUD software

$ export JAVA_HOME=/opt/oud/jdk
$ $JAVA_HOME/bin/java -d64 -jar /opt/oud/bits/fmw_12.2.1.4.0_oud.jar -silent -ignoreSysPrereqs -responseFile /opt/oud/bits/oud12c-standalone.rsp -invPtrLoc /opt/oud/bits/oraInventory.loc


6. Patch OUD to current bundle patch level

Set ORACLE_HOME environment variable

$ export ORACLE_HOME=/opt/oud/mw_oud12c


List patch inventory to see current OUD version and what patches are installed

$ $ORACLE_HOME/OPatch/opatch lsinventory


Install the OPatch patch

$ cd /opt/oud/bits/6880880
$ $JAVA_HOME/bin/java -jar /opt/oud/bits/6880880/opatch_generic.jar -silent oracle_home=$ORACLE_HOME


Install OUD patch responding interactively with y to both questions

cd /opt/oud/bits/35263333
$ $ORACLE_HOME/OPatch/opatch apply

List patch inventory to see compare with previous lsinventory output

$ $ORACLE_HOME/OPatch/opatch lsinventory


7. Set OPENDS_JAVA_ARGS environment variable so that when OUD instances start, they will use our custom tns-java.security configuration file rather than the default configuration file

$ export OPENDS_JAVA_ARGS="-Djava.security.properties=/opt/oud/bits/tns-java.security"

Note: Whenever you restart an OUD instance, you will want to set the OPENDS_JAVA_ARGS environment variable before starting the OUD instance.


8. Setup the first OUD instance

$ /opt/oud/mw_oud12c/oud/oud-setup --cli --integration eus --instancePath /opt/oud/mw_oud12c/oud1/OUD --adminConnectorPort 1444 --ldapPort 1389 --ldapsPort 1636 --httpAdminConnectorPort disabled --httpPort disabled --httpsPort disabled --baseDN dc=world --rootUserDN 'cn=Directory Manager' --rootUserPasswordFile /opt/oud/bits/.pw --addBaseEntry --generateSelfSignedCertificate --hostName $(hostname -f) --noPropertiesFile --no-prompt


9. Setup the second OUD instance

$ /opt/oud/mw_oud12c/oud/oud-setup --cli --integration eus --instancePath /opt/oud/mw_oud12c/oud2/OUD --adminConnectorPort 2444 --ldapPort 2389 --ldapsPort 2636 --httpAdminConnectorPort disabled --httpPort disabled --httpsPort disabled --baseDN dc=world --rootUserDN 'cn=Directory Manager' --rootUserPasswordFile /opt/oud/bits/.pw --addBaseEntry --generateSelfSignedCertificate --hostName $(hostname -f) --noPropertiesFile --no-prompt


10. Enable and initialize replication from OUD1 to OUD2

$ /opt/oud/mw_oud12c/oud2/OUD/bin/dsreplication enable --host1 $(hostname -f) --port1 1444 --bindDN1 'cn=Directory Manager' --bindPasswordFile1 /opt/oud/bits/.pw --replicationPort1 1989 --secureReplication1 --host2 $(hostname -f) --port2 2444 --bindDN2 'cn=Directory Manager' --bindPasswordFile2 /opt/oud/bits/.pw --replicationPort2 2989 --secureReplication2 --baseDN dc=world --adminUID admin --adminPasswordFile /opt/oud/bits/.pw --trustAll --no-prompt

11. Confirm that replication is working properly

$ /opt/oud/mw_oud12c/oud1/OUD/bin/dsreplication status --hostname $(hostname -f) --port 1444 --dataToDisplay compact-view --adminUID admin --adminPasswordFile /opt/oud/bits/.pw --advanced --trustAll --no-prompt

12. Configure OUD1 and OUD2 for TNS
Create /opt/oud/bits/tns.batch configuration file with the following content

set-connection-handler-prop --handler-name "LDAPS Connection Handler" --add ssl-cipher-suite:TLS_DH_anon_WITH_AES_256_GCM_SHA384 --add ssl-cipher-suite:TLS_DH_anon_WITH_AES_128_GCM_SHA256

create-password-policy --type generic --policy-name TNSAdmins --set password-attribute:userpassword --set default-password-storage-scheme:AES --set default-password-storage-scheme:Salted\ SHA-512

set-password-policy-prop --policy-name "Default Password Policy" --add default-password-storage-scheme:"EUS PBKDF2 SHA-512"

set-log-publisher-prop --publisher-name "File-Based Access Logger" --set enabled:true

Apply the eus.batch configuration file to OUD1

$ /opt/oud/mw_oud12c/oud1/OUD/bin/dsconfig -h $(hostname -f) -p 1444 -D 'cn=Directory Manager' -j /opt/oud/bits/.pw --batchFilePath /opt/oud/bits/tns.batch --no-prompt

Apply the eus.batch configuration file to OUD2

$ /opt/oud/mw_oud12c/oud1/OUD/bin/dsconfig -h $(hostname -f) -p 2444 -D 'cn=Directory Manager' -j /opt/oud/bits/.pw --batchFilePath /opt/oud/bits/tns.batch --no-prompt

Set OPENDS_JAVA_ARGS to use custom java.security and then restart OUD instances to apply cryptographic changes

$ export OPENDS_JAVA_ARGS="-Djava.security.properties=/opt/oud/bits/tns-java.security" 

Stop OUD1 and OUD2

$ /opt/oud/mw_oud12c/oud1/OUD/bin/stop-ds
$ /opt/oud/mw_oud12c/oud2/OUD/bin/stop-ds

Start OUD1 and OUD2

$ /opt/oud/mw_oud12c/oud1/OUD/bin/start-ds
$ /opt/oud/mw_oud12c/oud2/OUD/bin/start-ds


13. Create realm configuration to add TNS admin and grant privileges to manage database entries.  Here is a sample realm configuration file in LDIF format

dn: ou=TNSAdmins,cn=OracleContext
changetype: add
objectClass: top
objectClass: organizationalUnit
ou: TNSAdmins

dn: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext
changetype: add
objectClass: top
objectClass: organizationalperson
objectClass: inetorgperson
uid: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext
cn: tnsadmin
sn: TNS
givenName: Admin
userPassword: Oracle123
ds-privilege-name: password-reset
ds-privilege-name: unindexed-search
ds-privilege-name: modify-acl
ds-pwp-password-policy-dn: cn=TNSAdmins,cn=Password Policies,cn=config

dn: cn=Common,cn=Products,cn=OracleContext
changetype: modify
replace: orclSubscriberSearchBase
orclSubscriberSearchBase: dc=world

dn: cn=Common,cn=Products,cn=OracleContext
changetype: modify
replace: orclSubscriberNickNameAttribute
orclSubscriberNickNameAttribute: dc

dn: cn=Common,cn=Products,cn=OracleContext
changetype: modify
replace: orclDefaultSubscriber
orclDefaultSubscriber: dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonUserSearchBase
orclCommonUserSearchBase: ou=People,dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonUserCreateBase
orclCommonUserCreateBase: ou=People,dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonDefaultUserCreateBase
orclCommonDefaultUserCreateBase: ou=People,dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonGroupCreateBase
orclCommonGroupCreateBase: ou=Groups,dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonDefaultGroupCreateBase
orclCommonDefaultGroupCreateBase: ou=Groups,dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonGroupSearchBase
orclCommonGroupSearchBase: ou=Groups,dc=world

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonNicknameAttribute
orclCommonNicknameAttribute: uid

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonKrbPrincipalAttribute
orclCommonKrbPrincipalAttribute: userPrincipalName

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonNamingAttribute
orclCommonNamingAttribute: cn

dn: cn=Common,cn=Products,cn=OracleContext,dc=world
changetype: modify
replace: orclCommonWindowsPrincipalAttribute
orclCommonWindowsPrincipalAttribute: samAccountName

dn: cn=OracleContextAdmins,cn=groups,cn=OracleContext,dc=world
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleDomainAdmins,cn=OracleDefaultDomain,cn=OracleDBSecurity,cn=Products,cn=OracleContext,dc=world
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=PolicyCreators,cn=Policies,cn=LabelSecurity,cn=Products,cn=OracleContext,dc=world
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleDBCreators,cn=OracleContext
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleNetAdmins,cn=OracleContext
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleContextAdmins,cn=Groups,cn=OracleContext
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleUserSecurityAdmins,cn=Groups,cn=OracleContext
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleNetAdmins,cn=OracleContext,dc=world
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

dn: cn=OracleDBCreators,cn=OracleContext,dc=world
changetype: modify
add: uniqueMember
uniqueMember: cn=tnsadmin,ou=TNSAdmins,cn=oracleContext

14. Apply the realm configuration changes with ldapmodify

$ /opt/oud/mw_oud12c/oud/bin/ldapmodify -h $(hostname -f) -Z -X -p 1636 -D 'cn=Directory Manager' -j /opt/oud/bits/.pw -c -f /opt/oud/bits/tns.realm



Configure Database

1. Configure an Oracle database to register it into OUD by updating sqlnet.ora and adding ldap.ora
Sample sqlnet.ora illustrating the insertion of LDAP into the NAMES.DIRECTORY_PATH variable

NAMES.DIRECTORY_PATH= (LDAP,TNSNAMES, EZCONNECT)


Sample ldap.ora

DIRECTORY_SERVERS= (ol8.example.com:1389:1636,ol8.example.com:2389:2636)
DEFAULT_ADMIN_CONTEXT = "dc=world"
DIRECTORY_SERVER_TYPE = OID


2. Register Oracle database

Set Oracle database environment

$ export ORACLE_HOME=/opt/ods/poc/db/19c/dbhome_1
$ export ORACLE_BASE=/opt/ods/poc/db/19c/app/oracle
$ export TNS_ADMIN=/opt/ods/poc/db/19c/dbhome_1/network/admin
$ export ORACLE_SID=ol8


Register CDB database

$ /opt/ods/poc/db/19c/dbhome_1/bin/dbca -silent -configureDatabase -sourceDB $ORACLE_SID -registerWithDirService true -dirServiceUserName cn=tnsadmin,ou=TNSAdmins,cn=oracleContext -dirServicePassword Oracle123 -walletPassword Oracle123


Register PDB database

$ $ORACLE_HOME/bin/dbca -silent -configurePluggableDatabase -pdbName PDB1 -sourceDB $ORACLE_SID -registerWithDirService true -dirServiceUserName cn=tnsadmin,ou=TNSAdmins,cn=oracleContext -dirServicePassword Oracle123 -walletPassword Oracle123


3. Confirm registration with tnslookup

Lookup container database (CDB)

$ $ORACLE_HOME/bin/tnsping $ORACLE_SID

Lookup pluggable database (PDB)

$ $ORACLE_HOME/bin/tnsping PDB1_$ORACLE_SID

List all registered databases

$ /opt/oud/mw_oud12c/oud/bin/ldapsearch -T -h $(hostname -f) -p 1636 -X -Z -D cn=tnsadmin,ou=TNSAdmins,cn=oracleContext -j /opt/oud/bits/.pw -b dc=world -s sub '(|(objectClass=orclDBServer)(objectClass=orclNetService))'


I hope that you found this helpful!

Blessings!

Brad











Posted by at No comments:

Monday, October 16, 2023

OUD Feature Highlight: EUS/CMU Password Storage Scheme

Beginning with Oracle Unified Directory (OUD) 12c Patch Set 4, Oracle began adding new features and functionality along with bug fixes with each bundle patch release.  The What's New section of the documentation covers new features as they are added with each bundle patch release.

One such feature enhancement introduced a new password storage scheme that is used by Enterprise User Security (EUS) and Centrally managed Users (CMU) architectures for password based user authentication to the Oracle database. 

This new password scheme is a proprietary blend of multiple rounds of PBKDF2 SHA-512, which is much stronger than the storage scheme used for earlier Oracle database versions (e.g. 10g and 11g).  The full list of password storage schemes offered by OUD 12cPS4 are available here.

With the EUS architecture configuration where OUD is the identity store for authentication, identity management solutions like Oracle Identity Manager, SailPoint and others can simply update the password using normal password update and OUD password policy will automatically generate this password hash for database authentication.

With the EUS or CMU architectures where Active Directory (AD) is the identity store, the individual user's orclCommonAttribute value needs to be updated with this new hash in order for password based authentication to work properly.

The standard method of updating the user's orclCommonAttribute attribute value is through the deployment of the password filter to all AD domain controllers.  When a user updates their password with Ctrl-Alt-Delete feature of Windows, the Oracle's password filter (orapwdfltr.dll) captures the clear text password that was entered by the user, hashes the password, and stores the hashed value into the orclCommonAttribute attribute of the user's AD object.  See Doc ID 2640135.1 for more information on how to obtain and deploy the latest version of this password filter.

There is an alternative approach to populating the user's orclCommonAttribute in AD that has the same end result but does not require the password filter.  You can use the OUD encode-password command to generate the hashed value of the password and then update orclCommonAttribute in the user's AD entry.  This approach could be dove tailed into your provisioning solution as well.  Here is a sample workflow:

1. Install OUD.  
Note: If integrating with Identity Management solution, OUD would most likely need to be installed on the host(s) where the Identity Management solution is running in order to securely handle the password.

2. Use the OUD encode-password command to generated the hash of the user's password.

$ export JAVA_HOME=/opt/ods/poc/sw/jdk1.8.0_381
$ mkdir /opt/ods/poc/mw_oud12c/oud/locks
$ echo Johns-N3w_P4ssw0rd > /opt/ods/poc/cfg/...pw
$ /opt/ods/poc/mw_oud12c/oud/bin/encode-password -s MR-SHA512 -f /opt/ods/poc/cfg/...pw
Encoded Password: "{MR-SHA512}vGhKrQ39OEvg9vSOCJzm1TA/Eues1RN37ra+1rOuf6hfAPTFLd00CIVihRZ279OXNYbIEl2G/bjdaqOKxnuaye6rVgZcbdAjSZ9CTweXihU="


3. Use ldapmodify or equivalent API to update the orclCommonAttribute of the user's entry in AD.

$ /opt/ods/poc/mw_oud12c/oud/bin/ldapmodify -h msad.example.com -p 636 -X -Z -D 'cn=Administrator,cn=Users,dc=example,dc=com' -j /opt/ods/poc/cfg/...pw <<EOF
dn: cn=John Doe,cn=Users,dc=example,dc=com
changeType: modify
replace: orclCommonAttribute
orclCommonAttribute: {MR-SHA512}vGhKrQ39OEvg9vSOCJzm1TA/Eues1RN37ra+1rOuf6hfAPTFLd00CIVihRZ279OXNYbIEl2G/bjdaqOKxnuaye6rVgZcbdAjSZ9CTweXihU=
EOF
Processing MODIFY request for cn=John Doe,cn=Users,dc=example,dc=com
MODIFY operation successful for DN cn=John Doe,cn=Users,dc=example,dc=com


4. Test that the authentication works with the updated password.

$ORACLE_HOME/bin/sqlplus jdoe/Johns-N3w_P4ssw0rd@$ORACLE_SID


I hope that you found this helpful!

Blessings!

Brad
    Posted by at No comments:

    EUS: Troubleshooting ORA-01017

    Enterprise User Security (EUS) is one of the architectures available for centralizing authentication, authorization, and user/password lifecycle management or Oracle Database users.  One Oracle database error that customers often encounter as they begin evaluating the EUS architecture is the following:

    ORA-01017: invalid username/password; logon denied

    This error can be especially frustrating because there are a variety of possible causes.

    Here are some common causes and corresponding solutions and troubleshooting techniques:
    • The wallet wallet containing the ORACLE.SECURITY.DN and ORACLE.SECURITY.PASSWORD entries does not exist

    $ ls -al $ORACLE_BASE/admin/$ORACLE_SID/wallet

    • The wallet containing the ORACLE.SECURITY.DN and ORACLE.SECURITY.PASSWORD exists but is empty or has missing or incorrect values including case sensitive passwords.  To troubleshoot, retrieve the values from the wallet with:

    $ $ORACLE_HOME/bin/mkstore -wrl $ORACLE_BASE/admin/$ORACLE_SID/wallet -list -viewEntry ORACLE.SECURITY.DN -viewEntry ORACLE.SECURITY.PASSWORD <<EOF
    YourWalletPassword
    EOF

    • If Oracle database was upgraded from earlier version to 18c or newer, the mappings may need to be re-created. See Doc ID 2611300.1
    • The EUS configuration (e.g. Sample in /<oud_install/oud/config/EUS/modifyRealm.ldif) has not yet been applied or is mis-configured.  See Doc ID 2118421.1
    • The Certificate Authority (CA) certificate chain or OUD self-signed certificate is not loaded into the wallet.  To troubleshoot this issue, confirm the presence of the certificate in the wallet with:

    $ORACLE_HOME/bin/orapki wallet display -wallet $ORACLE_BASE/admin/$ORACLE_SID/wallet -pwd YourWalletPassword

    • Database start fails with ORA-01017. In this case grid user's group needs to be a member of the  OSRACDBA group.  See Doc ID 2313555.1
    • Get ORA-01017 with RAC database.  This can be caused by the having inconsistent wallets on each RAC node or by using the same wallet via NFS share on all three nodes but where auto_login only works for the node on which it was set.
    • May have specified the wrong ORACLE_SID environment variable value and the authentication fails because you are attempting to connect to the wrong database.
    • If using tnsnames.ora, the connect string may be pointing to the wrong database for which the user or user/password combination are not valid.

    When troubleshooting error ORA-01017 from the database perspective, you will want to enable tracing to determine the reason for the authentication failure.
      
    Step 1: Enable Oracle database tracing by with:

    $ $ORACLE_HOME/bin/sqlplus / as sysdba
    SQL> alter system set events '28033 trace name context forever, level 9';

    Step 2: Perform authentication attempt that fails with ORA-01017     

    $ $ORACLE_HOME/bin/sqlplus -S -L <eus_user_id>/YourUsersPassword@$ORACLE_SID
    ERROR: ORA-01017: invalid username/password; logon denied

    Step 3: Disable Oracle database tracing with:     

    $ $ORACLE_HOME/bin/sqlplus / as sysdba
    SQL> alter system set events '28033 trace name context off';

    Step 4: Lookup the path of the trace files (in case they aren't in default location):

    $ $ORACLE_HOME/bin/sqlplus / as sysdba
    SQL> sho param dbug;

    Step 5: Review trace file looking for KZLD_ERR messages
      

    When troubleshooting error ORA-01017 from the directory service perspective, you will want to review the directory service logs.  In the case of Oracle Unified Directory (OUD), you will want to review the /<oud_instance>/OUD/logs/access or /<oud_instance>/OUD/logs/access.log log file depending on which logger is enabled.  Things to look for include:
    • Authentication attempt by <eus_user_id> fails because user does not exist (err=32)
    • Authentication attempt by <eus_user_id> fails because the wrong password is used (err=49)
    • Connection to the OUD instances fails because of inability to  come to agreement on the LDAPS cryptographic negotiation.  Typically see error "no cipher suites in common". See Doc ID 2397791.1 for OUD 12c and Doc ID 2304757.1 for OUD 11g.  Note that this can happen if you've upgraded the JDK 8 to a version that has deprecated use of anonymous and NULL cipher suites.  In this case, you will need to update the jre/lib/security/java.security of the JDK implementation used by OUD to remove anon from jdk.tls.disabledAlgorithms.  Here is a sample java.security for jdk1.8.0_361:
    jdk.disabled.namedCurves = secp112r1, secp112r2, secp128r1, secp128r2, \
        secp160k1, secp160r1, secp160r2, secp192k1, secp192r1, secp224k1, \
        secp224r1, secp256k1, sect113r1, sect113r2, sect131r1, sect131r2, \
        sect163k1, sect163r1, sect163r2, sect193r1, sect193r2, sect233k1, \
        sect233r1, sect239k1, sect283k1, sect283r1, sect409k1, sect409r1, \
        sect571k1, sect571r1, X9.62 c2tnb191v1, X9.62 c2tnb191v2, \
        X9.62 c2tnb191v3, X9.62 c2tnb239v1, X9.62 c2tnb239v2, X9.62 c2tnb239v3, \
        X9.62 c2tnb359v1, X9.62 c2tnb431r1, X9.62 prime192v2, X9.62 prime192v3, \
        X9.62 prime239v1, X9.62 prime239v2, X9.62 prime239v3, brainpoolP256r1, \
        brainpoolP320r1, brainpoolP384r1, brainpoolP512r1

    jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
        DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, NULL, \
        include jdk.disabled.namedCurves

    jdk.tls.legacyAlgorithms= \
            K_NULL, C_NULL, M_NULL, \
            DH_anon, ECDH_anon, \
            RC4_128, RC4_40, DES_CBC, DES40_CBC, \
            3DES_EDE_CBC

    I hope you find this helpful.

    Blessigns!

    Brad




    Posted by at No comments:

    OUD Feature Highlight: Connection Details

    Beginning with Oracle Unified Directory (OUD) 12c Patch Set 4, Oracle began adding new features and functionality along with bug fixes with each bundle patch release.  The What's New section of the documentation covers new features as they are added with each bundle patch release.

    One such feature enhancement introduced connection details to each of the OUD log publishers.  When enabled, this enhancement tags each operation in the log file with the following additional details:
    • Authentication Distinguished Name:  bindDN=<user_dn>
    • Protocol: protocol=<LDAP | LDAPS | HTTP>
    • Client: client=<source_ip>:<source_port>
    • Server: server=<destination_ip>:<destination_port>
    • Cryptographic Protocol: protocol=<TLSv1.3 | TLSv1.2 | TLSv1.1 | ...>
    • Cryptographic Cipher Suite: cipherSuite=<TLS_AES_128_GCM_SHA256 | ...>
    These additional details can enable logging analytics tools like Oracle Cloud Infrastructure Logging Analytics to provide deep insights into the use and security posture of the OUD service to help you to strengthen security posture, identify out-of-date clients, identify root cause of cryptographic communication breakdowns, and even identify potential threat actors.  

    Here are some of the questions that each of these additional details can enable you to answer:
    • Authentication Distinguished Name
      • What operations did James perform on the directory server over the past 48 hours?
      • What users added new entries over the past 90 days?
      • Are any clients connecting anonymously to OUD?
    • Protocol
      • What clients or users are connecting to OUD via non-encrypted LDAP?
      • What clients are connecting via REST/SCIM?
      • Client
        • What is the volume of load per client IP address?
        • From what client IP addresses were write operations performed?
        • From what client IP addresses where anonymous authentications performed?
      • Server
        • Is the distribution of load even across servers in the load balanced pool?
        • What OUD instances receive write operations?
        • Which OUD instances are processing un-indexed searches or other abusive loads?
        • Which OUD instances are receiving non-encrypted LDAP connections?
      • Cryptographic Protocol
        • Which users are requesting weak cryptographic protocols like SSLv3?
        • What is the distribution of cryptographic protocols handled by the OUD service?
        • Based on client load, are we to disable weak cryptographic protocols?
        • Which clients need to be patched or updated to use strong cryptography?
        • Which clients need their trust store updated with the latest certificate authority certificate chain or perhaps need the updated self-signed certificates?
      • Cryptographic Cipher Suite
        • Which users are using anonymous or weak cipher suites when connecting to OUD?
        • What is the distribution of cipher suites being used by clients?
        • Based on client load, are we to disable weak cryptographic cipher suites?

      Enabling these additional connection details is very straight forward and can be enabled via the dsconfig command line (interactively, non-interactively, and in batch) tool or the web-based administrative console (Oracle Unified Directory Services Manager).  

      Here is a sample non-interactive dsconfig command that for enabling connection details to the File-based Access logger:

      dsconfig set-log-publisher-prop --publisher-name 'File-Based Access Logger --set log-connection-details:true --hostname $(hostname -f) --port 1444 --bindDN 'cn=Directory Manager' --bindPasswordFile /opt/ods/poc/cfg/...pw --trustAll --no-prompt


      Here is the equivalent batch configuration entry: 

      set-log-publisher-prop --publisher-name 'File-Based Access Logger --set log-connection-details:true 


      Here is a an access log excerpt with connection details disabled:


      [16/Oct/2023:15:48:10 +0000] CONNECT conn=42 from=10.20.0.104:59560 to=10.20.0.104:1636 protocol=LDAPS
      [16/Oct/2023:15:48:10 +0000] BIND REQ conn=42 op=0 msgID=1 type=SIMPLE dn="uid=admin1,ou=Admins,dc=example,dc=com" version=3
      [16/Oct/2023:15:48:10 +0000] BIND RES conn=42 op=0 msgID=1 result=0 authDN="uid=admin1,ou=Admins,dc=example,dc=com" etime=1
      [16/Oct/2023:15:48:10 +0000] SEARCH REQ conn=42 op=1 msgID=2 base="dc=example,dc=com" scope=sub filter="(uid=user100001)" attrs="dn"
      [16/Oct/2023:15:48:10 +0000] SEARCH RES conn=42 op=1 msgID=2 result=0 nentries=1 etime=14
      [16/Oct/2023:15:48:10 +0000] UNBIND REQ conn=42 op=2 msgID=3
      [16/Oct/2023:15:48:10 +0000] DISCONNECT conn=42 reason="Client Disconnect"


      Here is a an access log excerpt with connection details enabled:

      [16/Oct/2023:15:49:41 +0000] CONNECT conn=45 from=10.20.0.104:58190 to=10.20.0.104:1636 protocol=LDAPS
      [16/Oct/2023:15:49:41 +0000] CONNECT CONN_DETAILS conn=45 tlsVersion=TLSv1.3 cipherSuite=TLS_AES_128_GCM_SHA256
      [16/Oct/2023:15:49:41 +0000] BIND REQ conn=45 op=0 msgID=1 client=10.20.0.104:58190 server=10.20.0.104:1636 protocol=LDAPS type=SIMPLE dn="uid=admin1,ou=Admins,dc=example,dc=com" version=3
      [16/Oct/2023:15:49:41 +0000] BIND RES conn=45 op=0 msgID=1 client=10.20.0.104:58190 server=10.20.0.104:1636 protocol=LDAPS bindDN=uid=admin1,ou=Admins,dc=example,dc=com result=0 authDN="uid=admin1,ou=Admins,dc=example,dc=com" etime=0
      [16/Oct/2023:15:49:41 +0000] SEARCH REQ conn=45 op=1 msgID=2 client=10.20.0.104:58190 server=10.20.0.104:1636 protocol=LDAPS bindDN=uid=admin1,ou=Admins,dc=example,dc=com base="dc=example,dc=com" scope=sub filter="(uid=user100001)" attrs="dn"
      [16/Oct/2023:15:49:41 +0000] SEARCH RES conn=45 op=1 msgID=2 client=10.20.0.104:58190 server=10.20.0.104:1636 protocol=LDAPS bindDN=uid=admin1,ou=Admins,dc=example,dc=com result=0
      nentries=1 etime=1
      [16/Oct/2023:15:49:41 +0000] UNBIND REQ conn=45 op=2 msgID=3 client=10.20.0.104:58190 server=10.20.0.104:1636 protocol=LDAPS bindDN=uid=admin1,ou=Admins,dc=example,dc=com
      [16/Oct/2023:15:49:41 +0000] DISCONNECT conn=45 reason="Client Disconnect"

      I hope you find this helpful.

      Blessings!

      Brad


      Posted by at No comments:

      Friday, September 29, 2023

      TIME_WAIT: Mitigating TCP/IP Connection Exhaustion

      TIME_WAIT is an incredible part of the TCP/IP stack that enables connections to linger until the client properly closes the connection. However, in some cases, the client does not close the connection properly or efficiently. This can result in TCP connections in the TIME_WAIT state to persist until the operating system purges them. For a busy system, this can result in a denial of service on the host because all available connections get tied up in the TIME_WAIT state.

      Having worked with many large-scale and high-performance systems through the years, I've seen this scenario play out many times. Fortunately, each operating system has its own way of optimizing for this scenario to minimize the impact.

      To determine if this is a problem on your host, track the number of connections in the TIME_WAIT state. For example, on UNIX/Linux and MacOS systems, you can count these connections with netstat:

      $ netstat -an|grep -c TIME_WAIT
      45564

      Here are TCP tunings per operating system that I have used to mitigate this issue:

      RedHat/Oracle Linux 8: /etc/sysctl.conf
      net.netfilter.nf_conntrack_tcp_timeout_time_wait=1
      net.ipv4.tcp_tw_reuse=1
      net.ipv4.tcp_fin_timeout=1

      RedHat/Oracle Linux 7: /etc/sysctl.conf
      net.netfilter.nf_conntrack_tcp_timeout_time_wait=1
      net.ipv4.tcp_tw_reuse=1
      net.ipv4.tcp_fin_timeout=1

      Solaris 10/11:
      ndd -set /dev/tcp tcp_time_wait_interval 30000

      Windows:
      Reduce TcpTimedWaitDelay from the default of 2 minutes (120 seconds) down to around 20 seconds

      HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
      —> TcpTimedWaitDelay": dword:00000028

      HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
      —> StrictTimeWaitSeqCheck: dword:00000001

      Notes for Windows:
       · Changing these values requires a reboot. Plan to do that out of your production hours.
       · TcpTimedWaitDelay is 2 minutes by default, even if the value is not present in the registry.
       · You must set the StrictTimeWaitSeqCheck to 0x1 or the TcpTimedWaitDelay value will have no effect.

      While changing this parameter the following important points needs to be considered:
      Changing these values requires a reboot. Plan to do that out of your production hours.
      TcpTimedWaitDelay is 2 minutes by default, even if the value is not present in the registry.
      You must set the StrictTimeWaitSeqCheck to 0x1 or the TcpTimedWaitDelay value will have no effect.

      References:
      TcpTimedWaitDelay - https://technet.microsoft.com/en-us/library/cc938217.aspx
      https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc731521(v=ws.10)#BKMK_setdynamicportrange
      https://support.microsoft.com/en-in/help/929851/the-default-dynamic-port-range-for-tcp-ip-has-changed-in-windows

      Posted by at No comments:
      Subscribe to: Posts (Atom)