DRAG AND DROP ROBUSTNESS (also IMPROVEMENTS, PERFORMANCE)
- Dragging and dropping with multiple attempts only for first collections
- For later collections, if simple drag and drop, then for speed: try drag and drop first time, if it fails, filecopy
- For complex drag and drop in later collections (where Runnable code is added): may have to just resort to filecopy, because if it fails it becomes too complex to test
SOLUTION: did separate drag n drop test and a couple of tuts with dragndrop. Remaining tutorials are mainly done with background file copy.


VERSIONS:
- Selenium (selenium-server jar, previously selenium-server-standalone jar):
We started with selenium-server-standalone-3.9.1.jar and currently use selenium-server-4.0.0-alpha-5.jar

Although 4.13 is the last version that supports Java 8, it doesn't work with the existing code. It's not apparent to me from the errors, which are in Selenium's code, how to fix it on our end (any configuring changes) to get it to work.
Earlier versions up to at least 4.1.4 had the same problem. 

selenium-server-4.0.0-alpha-7.jar works. So we could try shifting up to that at least to get the latest version that readily works with existing code.

- Assertj-swing (assertj-core jar, assertj-swing jar, assertj-swing-junit jar):
We started with assertj-core-3.8.0.jar, assertj-swing-3.8.0.jar, assertj-swing-junit-3.8.0.jar
It seems like we can tentatively try moving to newer versions 3.17.1. However:

Compilation on windows has been tricky: it appears to compile fine for a while if I empty the lib folder with all the jars. So it doesn't immediately seem to detect or take into account such changes, even after I delete tutorial-tests.jar and GSTestingUtil.jar or got a fresh terminal, though detection of Greenstone testing code changes has been enforced in our build.xml.

So what may look like it compiles and starts up fine may act up some time later.


--------
Browser Testing canvas elements (Images GPS Map, Phind, Collage - in order of easier to harder)
--------

First see
- Promising: https://blogs.terrificminds.com/mastering-canvas-handling-in-selenium-with-java
- Payware: https://applitools.com/resources/videos/canvas-selenium-java/
- https://qxf2.com/blog/selenium-html5-canvas-verify-what-was-drawn/
- old: https://chariotsolutions.com/blog/post/automated-testing-of-html5-canvas/
- https://stackoverflow.com/questions/30050551/how-to-automate-pie-charts-and-bar-graphs-using-selenium/44366022

Google for: Can I take a snapshot of the image that the canvas is showing
Google for: You want to do a screenshot of the canvas


(Probably want to turn off checkbox Scroll Through Places before screenshot)
checksum the dataURL:
In JavaScript (or Java) use md5
md5sum/md5hash/...

Have a collage option (startup param?) to not randomise, limit number of images (same ordering each time, e.g. 3 images)

new testCollage(CLnumber) function
stops animation, stops timer
Place at topleft, centre and bottomright
returns md5 of screenshot
# Take care of returning empty string if there are < 3 images in collage

Make selenium call the testCollage() function. See if there's an eval()


Can test the Java Application version: screenshot preconfigured layout and compare with baseline. Then we confirm application still works.



For phind: turn off anti-aliasing/hinting
control fonts

---------

ASK KATHY:
Downloading docs from the web (Webtudor) tutorial: why is citizens/index.html never processed?
And should we use a different website for downloading example as it's been several years since robits.txt and tudor site overhaul prevented us from grabbing nicer looking layout (with all images) from the tudor website.

- New collection browse format is of form: <link>title<br/><i>(source)</i></link>
If you press reset to default, it becomes <link>title</link><br/><i>(source)</i>
Notice closing </link> element has moved. Why?
See Tudor collection.

TO INVESTIGATE:
+ Why is backdrop collection no longer producing Description field in OAIServer (like in collect.done currently called collect)?

Does it need a restart of the server? Something worked, as rebuilding the backdrop collection from scratch was fine.

>>>> Rename collect to collect.done.
>>>> Swap back collect.new to collect. 


ANSWER: image-e still had the old VList ClassiferNode definition
		      	<gsf:template match="classifierNode[@classifierStyle = 'VList']">
				<td valign="top">
					<gsf:link type="classifier">
						<gsf:metadata name="thumbicon"/>
					</gsf:link>
				</td>
				<td valign="top">
					<gsf:displayText name="ImageName"/>:
					<gsf:metadata name="Image"/>
					<br/>
					<gsf:displayText name="Width"/>:
					<gsf:metadata name="ImageWidth"/>
					<br/>
					<gsf:displayText name="Height"/>:
					<gsf:metadata name="ImageHeight"/>
					<br/>
					<gsf:displayText name="Size"/>:
					<gsf:metadata name="ImageSize"/>
				</td>
			</gsf:template>

Should be:
		      <gsf:template match="classifierNode[@classifierStyle = 'VList']">
				<td valign="top">
					<gsf:link style="static" type="classifier">
						<gsf:icon type="classifier"/>
					</gsf:link>
				</td>				
			</gsf:template>
			

+ Dragging and dropping needs to be made more robust by checking files got dropped (enough to spot test first and last in a contiguous selection were dropped) or to loop dragNdrop on the same files until they are, giving up after MAX_DRAG_N_DROP_ATTEMPTS for now set to 5. I think it should work on the 2nd go, failing after the 5th would point to a larger underlying problem. Maybe that problem would be not waiting/pausing long enough after (multi) selecting files to drag n drop, or waiting after each selection or waiting longer to get the drag ready.

X ToDo: Need to have a method to close all the tabs we opened in firefox. Ideally from Java. Otherwise commandline scripts that run on linux, mac and windows.

+ I thought about it some more, and the real solution is that the previewing should not happen in a regular firefox tab, but that the firefox selenium driver should open its automated firefox browser window at the preview URL. This required changing GLI to Not Gatherer.spawnBrowwer(previewURL) if in testing mode, but to provide a getPreviewURL method that the GLI automated testing will open in the automated selenium browser.

+ Ask Dr Bainbridge 
download/DownloadProgressBar.java (enableCancelJob() AND mirrorComplete() methods) :
ASK Dr Bainbridge
synchronizing some methods by synchronizing on member var. But that var has a method called on it after switching to EDT.

For why synchronizing on it was necessary, see point 4 of commit message to https://trac.greenstone.org/browser/main/trunk/gli/src/org/greenstone/gatherer/download/DownloadProgressBar.java?annotate=blame

- Above may need testing (regular GLI) after fixes by Dr Bainbridge: test downloading with cancel/stop and check processes halted.

------------------
 Tutorial updates
------------------

+ All files: Make all references to ZipPlugin be ZIPPlugin (note casing).

ImagesGPS:

1. Commit https://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/doc.pm?rev=38753
is marked as untested and makes ex.LatShort disappear, which is needed for the Images-GPS tutorial to work and which implementing the GLI automated test for it picked up.

I don't know how to fix the commit as I don't know what was intended. I have locally put doc.pm to the previous revision of 38752 and know that works fine.

This needs fixing, Kathy may know what to do.

+ 2. Need emphasis on metadata, buttonname and locations in the following instruction of the Images-GPS tutorial: "In the configuration dialog that appears, set the metadata field to dc.Title and tick the buttonname option and set its value to locations."


Enhanced Tudor:

+ Put back the Phind instructions (see tutorial for GS 3.06)

? CL2 modification #2: need valign="top" on the td:
      <td valign="top">
      <gsf:link type="document">
      <gsf:metadata name="Title"/>
      </gsf:link>
      </td>
      
+ Multimedia:
Put back the Phind and Collage instructions or copy them from GS2 if they weren't yet in GS3.
(3.07 has Phind instructions)


bibliographic/MARC tutorial
+ "The ex.Source metadata is set to the nul filename, so remove that from the display. Once again, from BOTH browsing classifiers (browse and CL1 List), remove:"

Only refer to browse, there is no CL1

+ MARC tutorial instructions need to tell users to change search display for exp.Subject and exp.Title to subjects and titles, respectively.

+ Step 10: "form search" is now called "fielded search"


Multimedia coll

+ dc.Title;Title => dc.Title,Title

X After step 16 should be 17 instead of bullet point?

+ Phind in smallbea, Collage for large beatles

+ All of form <gsf:when test='equals' test-value='Audio'> should have "" (&quot;) not ''

X Is <gsf:link type="source"><gsf:metadata name="srcicon"/></gsf:link> better to split over 3 lines?

Scanned Images coll
+
<i>
<gsf:metadata name="Source"/>)
</i>

Needs to be all on the same line and need opening bracket.


In step 11:
+ "In the Format Features section of the Format panel, select DateList in the Choose Feature list, and click <Add Format> to add this format statement to your collection. In the documentNode template of the new DateList feature, replace:

<gsf:switch>
<gsf:metadata name="Source"/>
<gsf:when test="exists"/>
<br/>
<i>(<gsf:metadata name="Source">)</i>
</gsf:when>
</gsf:switch> "

Need to self-close the Source gsf:metadata element.


+ Step 19 (20): "form search" is now called "fielded search"


UnknownConverterPlugin tutorial
+ djvuxt -> djvutxt

TODO: Why are we not allowed to nest the exec_cmd parts in double quotes any more (UnknownConverterPlugin config)?


+ METS tutorial: ZIPPlugin not ZipPlugin

TODO: Does GLI exit on testing need explicit call to dispose()? Test with unknownPluginConverter tutorial


TODO: maybe the compulsory warning popup in the simplehtml() automated test function should be made optional, or a clearer message that it was compulsory.

TODO: ASK KATHY:
Small beatles -> phrase browser (applet) has about/home page image. What have I done wrong?


+ Indexers tutorial
Go to the fielded search page next, and once there, ENSURE the stem option is back to off (whole word must match) to avoid confusion later on.


+ Incremental Building tutorial
Change line:
       * 3 documents were processed and included in the collection" 
To:
	* 3 were processed and included in the collection

(Note double quotes at end and word "documents" are missing)



Customising Home Page tutorial
   Line
Couldn't find and/or load the stylesheet "pages/home-tutorial.xsl" for a=p, sa=home, in collection.
   Should be (note = at end)
Couldn't find and/or load the stylesheet "pages/home-tutorial.xsl" for a=p, sa=home, in collection=


+ Designing a new interface 2
Step 6: 

Regarding

<xsl:template name="additionalHeaderContent">
<script type="text/javascript" src="interfaces/{$interface_name}/scripts/jquery.min.js"><xsl:text> </xsl:text></script>
<script type="text/javascript" src="interfaces/{$interface_name}/scripts/jquery.easing.1.3.js"><xsl:text> </xsl:text></script>
<script type="text/javascript" src="interfaces/{$interface_name}/scripts/jquery.timers.1.2.js"><xsl:text> </xsl:text></script>
<script type="text/javascript" src="interfaces/{$interface_name}/scripts/jquery.galleryview.2.1.1.min.js"><xsl:text> </xsl:text></script>
<script type="text/javascript" src="interfaces/{$interface_name}/scripts/jquery.galleryview.setup.js"><xsl:text> </xsl:text></script>
</xsl:template> 


The first <script> above (inclusion of jquery.min.js), if by itself, causes the following error seen in console:
"Uncaught TypeError: $(...).button is not a function"
(see https://stackoverflow.com/questions/34530211/uncaught-typeerror-button-is-not-a-function)

Actually the correct solution for the final stages of the Designing A New Interface tutorials is to just comment out the first <script> above, because jquery.min.js is already referenced in header.xsl and this inclusion here (as it is by itself and not followed by jquery-ui) conflicts with jquery-ui which gets loaded earlier through header.xsl. The result is the error described above.
 However, if the first line (inclusion of plain jquery) is here commented out, then in this early step 6 of the 2nd Designing A new Interface tutorial, there is no jquery-ui yet and the page ends up devoid of the image slider. And that's a worse error for testing.

So the hack is to add a 2nd line below to refer to jquery-ui as follows: 
<script type="text/javascript" src="interfaces/{$interface_name}/js/jquery-ui-1.13.0.custom/jquery-ui.min.js"><xsl:text> </xsl:text></script>

This and the jquery inclusion will end up duplicated in the final version (as header.xsl refers to them too), but they need to be present for some intermediate steps, as described.
Maybe later in the tutorial, a step can explain that these 2 lines can be removed from the additionalHeaderContent template.


Themes tutorials
+ jquery-ui-1.11.4.custom -> jquery-ui-1.13.3.custom

+ 1.13.2 (stable ...) -> 1.13.3 (Legacy, for jQuery 1.8+)