Essential to Know When Installing HEI 11.0.1 on IBM i

HCL Enterprise Integrator (HEI), is more commonly known as Lotus Enterprise Integrator (LEI). You may also know it as IBM Enterprise Integrator (IEI). Despite the name changes, this product is just as amazing as it has always been. Regardless if Domino is one of the data stores or not, it synchronizes data between disparate systems. This means perhaps syncing data between DB2, Oracle, SQL Server, SAP, Domino, File Systems, … the list goes on. In addition this amazing product allows for virtual views and virtual fields when interfacing with Domino applications.

Enough reminiscing about functionality, what happens when you upgrade the product on the IBM i to version 11.0.1?

For those of us familiar with the product, we know the version of HEI needs to match the version of Domino. This does not change when upgrading to HEI 11.0.1. What does change is related to the Licensed Program Product for HEI when installing on the IBM i.

In prior versions, when installing HEI (well, technically LEI), the version of ‘LEI’ installed was removed and replaced with the new version. This is still true when installing HEI 11.0.1, but with a bit of a caveat. Even though there is a “success” message when installing HEI 11.0.1, when displaying the Licensed Program Products on the IBM i system, the big surprise is the product will still show as 5733LEI, not at all what one would expect!

As a result, the keys to ensuring LEI has been upgraded to version 11.0.1 is to validate the following:

1. Examine the contents of the ‘/qibm/proddata/lotus/domino110001’ directory. The following files should be present:
   1. lei.lic
   1. lei.pgm
   1. leiact.pgm
   1. leicsm.pgm
2. When starting the HEI server, the release level of HEI will be displayed and should show as being 11.0.1.

Happy Upgrading!

HCL Domino and Notes v12 are Coming!

I will be co-presenting at the webinar “HCL Domino and Notes v12 are Coming!” on April 22 at 10:00 AM EDT / 4:00 PM CEST. This webinar is co-sponsored by HCL and panagenda. Barry Rosen will share details on what is coming in version 12, with a primary focus on the Notes client. He will also share details on what is coming in future releases.

Christoph Adler and myself will talk about upgrading the Notes client. Christoph will share details on how to automate the upgrade process. I will focus on the customer experience in upgrading the Notes client. Key topics we will cover include:

• Auditing Existing Notes Client Deployments
• Centralized Tracking and Reporting for Client Upgrades
• Scheduling Methods for Upgrade Activities

You can register for the webinar here: https://info.panagenda.com/en/domino-and-notes-12-is-coming

I look forward to seeing you in our joint webinar!

In the event you missed the webinar, the recording can be found here: https://bit.ly/3vf3DVT and the slide deck can be found here: https://bit.ly/3nf1lmG

FP4 for Domino 10.0.1 on IBM i

Domino 10.0.1 has been available on the IBM i for a number of months now, however there have been no Fix Packs available for 10.0.1 on the platform to this point. That changes today, October 4 2019.

Many have been wondering where FPs for 10.0.1 on the IBM i have been. The reality is 10.0.1 for the IBM i has been 10.0.1.1 as the code load contains the fixes included in FP1. However, there have been no 10.0.1 FPs for the IBM i since, until today. This is due to issues HCL found in their internal testing that prevented them from releasing 10.0.1 FPs for the platform.

The issues are still being worked, so what HCL has done in order to get the FP4 code out to the IBM i community is create a full install for FP4. This means the prior 10.0.1 product needs to be deleted before FP4 can be installed. As is the case for the full product load, there are 3 savefiles associated with the installation.

• QNOTES – *BASE of the 10.0.1 product
• QNOTESAP – Option 1 which is the C API
• QNOTESRL – Option 11 which is 10.0.1 (with FP4)

A couple of things to be aware of with FP4 being a full installation rather than just installation of a Fix Pack:

• If you have Traveler installed with 10.0.1, you will need to reinstall Traveler after installation of FP4.
• If you have any language packs installed, they will need to be reinstalled after installation of FP4.

For the official technote regarding FP4 for the IBM i refer to KB0069162.

Details on the installation of FP4 are in KB0069486.

Sneak Peak at Domino 10.0.1 on IBM i

It’s coming … just around the corner … can’t wait!!!!!!

One of the biggest questions I have been getting is which IBM i operating systems will support Domino 10.0.1. You will be very pleased with the answer 🙂 The releases supported are: 7.2, 7.3, and 7.4 (when it becomes available). The huge relief is that 7.2 is supported, whew!

In addition to the plethora of functionality that is provided in release 10.0, 10.0.1 brings some unique enhancements that are specific to the IBM i, let’s take a look at what these are.

Daylight Saving Time enhancement

This first one is long overdue. It’s one that is very subtle, until daylight saving time hits, then wham!! In Domino 10.0.1 on the IBM i, it will default to *TIMEZONE when you configure a new server. This means no more times being off by one hour when DST sets in because you forgot to add the DSTLAW= variable to the notes.ini when you configured the server.

Easier control over JVM used by Domino

Another enhancement to make administrator’s lives easier is the ability to change which JVM is in use for a Domino server. By default the 32-bit JVM is used for Domino on the IBM i. The process of changing from the 32-bit to the 64-bit JVM has been to add the JAVA_HOME variable to the notes.ini. While this doesn’t sound like a big deal, it can be a bit tricky. Below is the JAVA_HOME variable to convert a Domino server to use the 64-bit Java 8 JVM.

  JAVA_HOME=/QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit

The main trick is case sensitivity, specifically with /QOpenSys. If you do not have the correct case for this portion of the path, it will not work. Domino 10.0.1 to the rescue! In this new release, there is a JVM version option available with both the Configure Domino Server (CFGDOMSVR) and Change Domino Server (CHGDOMSVR) commands.

MEMCHECK automatically collected in Domino 10.0.1

If you get into debug mode with your Domino servers, you will be elated that MEMCHECK is enabled by default on an IBM i Domino 10.0.1 server. You may be wondering why this is a big deal. Prior to Domino 10.0.1, this valuable debug data was not available unless you manually ran a memcheck on the IBM i, which very few customers know how to do.

Licensed Program Product (LPP) and Library changes

Other changes to be aware of are the licensed program product ID and library where the Domino 10.0.1 code resides. We are used to seeing 5733L85 and 5733LD9 for 8.5.3 and 9.0.1 respectively for the product ID. The 10.0.1 product will be 5733D10 with *BASE being the Domino 10.0 code and Option 1 being 10.0.1. We can quite easily guess that 10.0.2 will be Option 2.

The library where the Domino 10.0.1 code executables resides is probably the most noticeable difference. With libraries having a maximum length of 10 characters, the nomenclature of QDOMINO8xx and QDOMINO9xx no longer works with version 10.x. Therefore the library structure changes to QDOM100001 for version 10.0.1. The table below provides a summary of what the Domino releases and their respective LPP, library, and directory details look like for releases 8.5.3, 9.0.1 and 10.0.1.

Domino Release	LPP ID	Option	Program Library	Product Directory
8.5.3	5733L85	*BASE, 1, 13	QDOMINO853	/QIBM/ProdData/LOTUS/domino853
9.0.1	5733LD9	*BASE, 1, 11	QDOMINO901	/QIBM/ProdData/LOTUS/domino901
10.0.1	5733D10	*BASE, 1, 11	QDOM100001	/QIBM/ProdData/LOTUS/domino100001

When do we get all of this?!?

So now the big question remains, WHEN will Domino 10.0.1 be available on the IBM i?!?

I have some important discoveries to share on this front. In preparing for the Domino Technical University webcast I will be doing with Chris Adler on February 7th on Best Practices to Upgrading to Domino 10 (https://bit.ly/2BiMrVO), I found the following when reviewing the product documentation:

In addition when searching in Passport Advantage, I see this for Domino 10.0.1 for the IBM i.

When clicking the link to download it, this secret is revealed 🙂

I hope you are as excited as I am to get this phenomenal, feature rich, self-healing release available on the hardware platform that is most synonymous with Domino, the IBM i: it’s integrated, secure, scalable, and highly available.

I’ll leave you with some helpful links:

What’s New in Domino 10.0.1 for IBM i: https://www-01.ibm.com/support/docview.wss?uid=ibm10794261

IBM Domino 10.0.1 Software Product Compatibility Reports: https://ibm.co/2RBJkO6

Domino 10 and the IBM i

Many of my customers, colleagues and friends in the community have been asking me when and “if” Domino 10 is going to be available on the IBM i. The rate of questioning increased after the announcement that HCL is going to purchase Domino, progressing beyond the current partnership agreement between IBM and HCL.

At first I was a bit perplexed why there was more concern with the purchasing agreement vs. the partnership agreement. I asked some questions and the answers became obvious. People’s assumption was that the IBM i would be supported by IBM because it owns the system and HCL does not. While I completely understand this assumption, reality couldn’t be farther from the truth.

HCL has listened to ALL customers of the Domino platform and has realized that IBM i customers are very committed to the product and many use it to provide key components of their businesses. In addition, HCL gets what Domino really is, an application development platform. They don’t see it as just an email, calendar, and contacts solutions which sadly is how the ones in IBM who decide where $$ go see it; rather HCL sees the significant advantage this low-code platform provides to companies to give them a competitive advantage.

HCL is fully committed to the IBM i platform, in ways we have not seen since the early 2000s. I am under NDA, so I cannot say exactly when Domino 10 will be available on the IBM i, what I can safely say is it will be in the first quarter of 2019.

Stay tuned, it’s coming soon!!!

Self-Healing Capabilities of Domino 10 presented at CollabSphere

I did a session at CollabSphere on the Self-Healing Capabilities of Domino 10. The functionality added into Domino 10 for maintaining a healthy Domino environment is absolutely amazing.  My presentation covers 5 issues that plague administrators on a day-to-day basis, how these issues are handled today and how these issues are remedied in Domino 10.  The 5 issues I covered are:

• Issue 1: Missing replicas and NLOs
• Issue 2: Corrupt NSFs and NLOs
• Issue 3: Missing documents
• Issue 4: Critical views out of date
• Issue 5: Who Deleted my documents

If you utilize clustering in Domino, you will be applauding Automatic Cluster Database Symmetry.  This functionality ensures both NSFs and NLOs are kept in sync across cluster mates.  I have never worked with a customer environment that didn’t have missing NSFs on their cluster mates and I have found when DAOS is being used, inevitably there are at least a handful of NLOs missing on one or more servers.  Automatic Cluster Database Symmetry ensures these discrepancies are repaired with no administrator intervention required once things are setup.

What’s even more cool is that if NLOs are encrypted with the server’s ID, the NLOs are decrypted with the source server’s ID, repaired/replicated, and then encrypted with the target server’s ID, provided NLO encryption is enabled (which is the default setting).

There is a new Cluster Configuration document that allows administrators to select which cluster to work with.  Based on the chosen cluster, the Symmetrical Cluster tab of the new configuration document allows you to select which folders should be monitored by the server to maintain symmetry on.  This allows you to keep both NSF and DAOS content synchronized across a cluster.

I have uploaded the presentation to SlideShare: https://www.slideshare.net/KGCI/self-healing-capabilities-of-domino-10-108188426

I would like to thank the HCL Development team, in particular Gary Rheaume, for all of their input on the content of the presentation.  Gary was awesome and created two videos I was able to use during my presentation as demos.  The great news is both of these videos are now available on YouTube!

The first demo is of Automatic Cluster Database Symmetry and can be found here: https://www.youtube.com/watch?v=8q-AWayRxwI

The second demo is of Automatic Database Repair and can be found here: https://www.youtube.com/watch?v=IKYLJWpEWBo

I will be creating additional blog posts regarding these amazing features coming in Domino 10.

Installing FP9 on IBM i is Quite Different and Very SLOW

If you are planning to install FP9, there are a few things you need to be aware of:

1. The installation process is quite different from what you are used to for installing FPs on the IBM i
2. It’s significantly slower, as in hours, so plan accordingly

Installation process

The installation process is a fair bit different than installing previous FPs.  You no longer just download the savefile, FTP it so the server and then load and apply the associated PTF.  Instead, you download the savefile, FTP it to the server and then things get a fair bit different.

Step 1.

The first step after downloading the savefile, which is called KITFP9, is to restore the contents of the savefile. This savefile contains two savefiles:

 

 

Make sure you leave the default of restoring the objects into the saved library of QGPL rather than specifying a library of your own, because the install script that is used to do the installation assumes the savefile for FP9 (QFP69019) is in library QGPL.

The restore command provided is:

RSTOBJ OBJ(*ALL) SAVLIB(QGPL) DEV(*SAVF) SAVF(QGPL/KITFP9)

 

Step 2.

The next step is to restore the console installer from savefile QCLINSTF.

RST DEV(‘/QSYS.LIB/QGPL.LIB/QCSLINSTF.FILE’) OBJ((‘/HOME/QNOTES’ *INCLUDE *SAME)) CRTPRNDIR(*YES) PRNDIROWN(QNOTES)

This will restore 81 files into /home/qnotes.

The first level is the console_install directory which contains the install script, install.sh, along with the LAP directory.

 

 

 

The Readme notes to NOT restore the contents into /QIBM/PRODATA/LOTUS/DOMINO901 as this is the directory where the Domino 9.0.1 executables are stored, so either leave the default of /home/qnotes or specify another directory you know is not in use by other applications.

Step 3.

As normal, end all active Domino servers on the system or LPAR that are running Domino 9.0.1.

Step 4.

This is where things get a bit different.  Rather than just issuing LODPTF and APYPTF, instead, you invoke QShell and run the installation script that is in the /home/qnotes/console_install directory. So the steps become:

QSH

cd /home/qnotes/console_install

./install.sh

The installer at this point will issue a LODPTF and APYPTF, remember it will bomb out unless the QFP69019 savefile is in library QGPL. I haven’t had time to work with modifying the install script and testing to see if things work okay with the having the savefile in another library, so for now stick with QGPL.

Step 5.

You will next be presented with the following screen regarding the license agreement.

                                                        QSH Command Entry

This document includes License Information documents below
for multiple Programs. Each License Information document
identifies the Program(s) to which it applies. Only those
License Information documents for the Program(s) for which
Licensee has acquired entitlements apply.

  ==============================================

LICENSE INFORMATION

The Programs listed below are licensed under the following

Press Enter to continue viewing the license agreement, or
enter “1” to accept the agreement, “2” to decline it, “3”
to print it, or “99” to go back to the previous screen.

You will want to type in 1 and press Enter to continue.

Step 6.

Wait, and wait, and wait, and wait, …. yes continue to wait, because it’s going to be a LONG, LONG time before the installation completes.  More on that in the Installation performance section below.

Step 7.

Once the installation finally finishes, load the most recent Interim Fix or custom hotfix if you have one from IBM.

In this case I installed IF2, so the commands were:

LODPTF LICPGM(5733LD9) DEV(*SAVF) SELECT(L605552) SAVF(GREENK/QL605552)

APYPTF LICPGM(5733LD9) SELECT(L605552)

Step 8.

Start the Domino servers and test.

Installation performance

The Readme states this: Domino 9.0.1 FP includes Dojo version upgrade with thousands of stream
files, need to plan for additional time required to copy these stream files to
Domino 9.0.1 product data and each server data folders during the installation
process.

The Readme for FP8 had this same statement, and the process was fairly quick, it is NOT for FP9, so what’s different?

From what I have observed, the big difference is how they handle copying the thousands of stream files.  In FP9, for every stream file that is copied, there are 3 messages that are generated and written to QShell in addition to be written to a spooled file:

1. CPCA087 which acknowledges the location the object was copied from and written to.
2. The first CPC221B does a Change Authority on the object that was copied.
3. The second CPC221B does a Change Owner on the object that was copied.

Below is an example for one stream file.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Not only are these 3 messages generated for the THOUSANDS of stream files it copies, the files are copied to every Domino server installed on that system or LPAR!!! None of this is done in parallel, the files are copied to the first server, then the next, and the next …  No wonder it takes FOREVER!

The system this installation was done on is an IBM Power System S814 with 3.72 GHz chip speed, 68 GB of memory and over 10 Terabytes of disk.  Needless to say there were ample resources available on this system to handle the workload of the installation. There are 4 Domino servers installed on this LPAR.

Over 3 hours after I started the installation of FP9, it finished.  This is unheard of.  When I installed FP8 on this system, it was done in around 10 minutes.

I generated a spooled file once the installation completed (pressed F6 in QShell).

The spooled file that was generated was 3,515 pages in length, and only contains output from the timeframe of 17:03:17 – 17:04:35.

You think someone forgot to turn off the debug code before this was shipped?!?

This problem has been reported to IBM with hopes the performance can be significantly improved.

Want Verse on Prem & Connections 6 on IBM i and AIX? Then Vote!

I have spent a fair amount of time this past year working with various groups within IBM to get Verse on Premises (VoP) supported on the IBM i. The net of these discussions is in order to get Verse on Premises supported on the IBM i or AIX platforms, Docker will be required. Docker is a technology that allows for applications to be created, deployed and run using containers.  By using Docker, applications and services can be built and deployed on any operating system that supports Docker.

In addition to Verse on Premises needing Docker, Connection 6 (PINK) will also require Docker.  In order to get these two collaboration solutions supported on IBM i and AIX, I have created two Requests For Enhancements (RFEs):

• Docker on IBM i, RFE # 112795
• Docker on AIX, RFE # 112797

I know both the IBM i and AIX communities want to have VoP and Connections PINK supported on their respective platforms.  The key is that the port of Docker to the IBM i will start with the port of Docker on AIX, so I am asking that you please vote for both of these RFEs to ensure both platforms become Docker enabled.

In addition to voting for the RFEs, adding comments to the RFEs will be very helpful to IBM in evaluating the importance of having Docker supported on these platforms.  By sharing why these platforms are important to you and how these collaboration solutions will benefit your business, IBM will have more knowledge when having discussions about these RFEs.

Please vote!!

 

Domino on the IBM i (iSeries) IS supported!

I received an email today from one of my customers inquiring about this blog post: http://domino.elfworld.org/ibm-connect-2017-3-i-ve-seen-the-future-of-domino-and-it-is-sapho/

Their question to me was:

“Just a quick question, I was reading posts about Connection 2017 and came across this about halfway down.

Other news:
No more Notes client for Linux beyond 9.0.1 FP 7
32 bit droppet for AIX and Linux servers
No more Domino for iSeries

Is this true?”

I was at IBM Connect and had not heard this message and had previously been told that Domino on the IBM i would be supported along with all other currently supported platforms.  As a result, I reached out to Barry Rosen, the Offering Manager for IBM Collaboration Solutions.  He confirmed that Domino on the IBM i IS indeed supported.  He also posted the following comment on the blog post (thank you Barry!)

and the blog post has been updated to remove this statement (thank you Hogne).

The IBM Domino roadmap https://www.ibm.com/blogs/social-business/2016/09/12/ibm-notes-domino-v9-extends-support/ states “IBM Notes and Domino V9.0.1 is extending support through at least 2021”.  In my correspondences with Barry last October, I also received this statement in an email from Mr. Rosen that clearly states Domino on the IBM i is supported:

“Domino 9.0.1 on IBM i will be supported through at least 2021.  All current supported platforms will be supported through at least 2021.”

As I told my customer, rest assured, Domino on IBM i is fully supported, along with all feature packs.

Protecting your Domino servers from the clickjacking hack

There is a hack called clickjacking that can happen on web servers, including Domino.  Here are the details on how clickjacking can impact web sites.

An attacker performs a clickjacking attack by creating a site on the Internet, which contains inline frames (iframes) that can display content from the application.  The attacker sets the malicious iframes as invisible and places them on top of a commonly clicked link or icon found on the webpage.  Using JavaScript-based functions and other techniques, the attacker can force the authenticated user to click on and unknowingly execute target application functions. This exploit could control user’s actions without their knowledge and could potentially enable an attacker to expose confidential information or impersonate users.

For example let’s say users connect to the mail server via the URL https://mail.companyxyz.com.
This site can be included on a webpage with an iframe containing the following  <iframe src=”https://mail.companyxyz.com/” width=”500″ height=”500″></iframe>

The way you mediate this hack depends on the release level of the Domino server.

For any servers running 9.0.1 FP6 or higher, the following notes.ini variable can be set.  It just requires an end and restart of HTTP for this change to take effect.

HTTPAdditionalRespHeader=X-Frame-Options: SAMEORIGIN

For servers running earlier versions of Domino, those servers can be switched to use Internet Sites documents and then a Web Site Rule can be created that specifies a custom header with the x-frame-options header set to SAMEORIGIN.

If you haven’t enabled your server to use Internet Sites, edit the server document and specify “Enabled” for field ‘Load Internet configurations from Server\Internet Sites documents’.

Next create a Web Internet Site document, specifying the values appropriate for your site.  In the Web Site document, click Web Site -> Create Rule, select “HTTP response headers” for the ‘Type of rule’.  Under ‘Custom headers’, enter “X-Frame-Options” for the Name and “SAMEORIGIN” for Value and place a checkmark next to “Override“.

Whether you have enabled the notes.ini variable on a 9.0.1 FP6 or higher server or enabled the capability through a Web Site Rule in an Internet Site document, end and restart the HTTP task for prevention of clickjacking on your Domino server to be enabled.

Here is a technote for reference: http://www-01.ibm.com/support/docview.wss?uid=swg21568598