SOLVED: Perl upgrade from 5.26 to 5.28 breaks Perl environment

[xamaco]

I run a big Perl environment with many CPAN moduls partly installed from downloaded moduls, partly installed by the MCPAN program.

Upgrading Perl from 5.26 to 5.28 via APTus did not create the related directory 5.28.0 in directory /usr/local/lib/x86_64-linux-gnu/perl. Thus Perl didn't know anything other then Perl itself, all moduls vanished.

As a first workaround I copied the contents of my 5.26.2 directory into a newly created 5.28.0 directory.

But there is still another issue, which I cannot solve. One of my Perl programs, running already a long time without problems, now lacks a function throwing out this message:

In mathoms.c, line 1156, starting in Perl v5.30, to_utf8_lower() will be removed. Avoid this message by converting to use toLOWER_utf8_safe().

It does not end my program, but the desired feature simply does not work. So I think, the mathoms.c problem (mathoms.c is a c-library providing slightly deprecated functions; since I cannot find any mathoms.c file on my machine, I suppose it is compiled into the Perl binary) causes this issue (The same function in my program, using case sensitivity, works well). I really do not know, wether this is a packaging or rather a genuine perl problem.

I would appreciate very much any help with this issue.

[paxmark1]

I do not run any CPAN modules

Code [Select]

http://sparkylinux.org/forum/index.php/topic,4651.msg11879.html#msg11879  does not affect me that much. 

I believe that you are going to not get much help here - a lot more hobbyists than serious coders here. 

Personally, I always use the command line for my upgrades.  It is not so much that I do not trust Aptus, but that I view the process in 3 stages in a large terminal. It slows me down and lets me think.    And I watch the Debian Transitions page every few days, they hit Sid at least 5 days before. 

Code [Select]

https://release.debian.org/transitions/html/perl.html

I could be very wrong - but I believe
dpkg, apt and apt-get will always put the Debian packages in /usr.  I am guessing that CPAN puts its Perl into /usr/local.  (Unless it goes into /opt)

I am going to guess but the

In mathoms.c, line 1156, starting in Perl v5.30, to_utf8_lower() will be removed. Avoid this message by converting to use toLOWER_utf8_safe().

  appears to be a warning statement about an intended change in Perl occurring at the next transition to 5.30. 

If the final output of your process in incorrect, it might be something else causing a problem. If that is the only line of warning that appears, you might be getting valid results.  I do no know. 

  I am not a Perl guru.  I would recommend looking for an IRC that is focused on CPAN or finding people who use CPAN perl locally. 

Hopefully I am proved wrong and a Perl guru gives a short reply to fix.  Good luck.

[xamaco]

Hi there,

first of all I want to thank paxmark1 for his fast and informative reply. After looking around in several source files of Perl and Perl/Tk it was also this post, which motivated me to search the fault in my code. What a nag, since I had nearly 9000 lines of code.

I learned, there is a UNIX utility, which was not present in SparkyLinux. After installing package 'moreutils' I had a command line statement named 'isutf8' (usage: isutf8 <FileName>), which tells me, if my code file (the file where I discovered the bug) is properly encoded in UTF-8.

Unfortunately my source file went through this test. Since I had other ones from older times, I could test the isutf8 statement with them, so I finally knew, the tool is OK.

My error message was:

Malformed UTF-8 character: \xa6 (unexpected continuation byte 0xa6, with no preceding start byte) in subroutine entry at /usr/local/lib/x86_64-linux-gnu/perl/5.28.0/Tk.pm line 251.
Tk::Error: Malformed UTF-8 character (fatal) at /usr/local/lib/x86_64-linux-gnu/perl/5.28.0/Tk.pm line 251.
Tk::__ANON__ at /usr/local/lib/x86_64-linux-gnu/perl/5.28.0/Tk.pm line 251
Tk::Derived::Delegate at /usr/local/lib/x86_64-linux-gnu/perl/5.28.0/Tk/Derived.pm line 469
Tk::Widget::__ANON__ at /usr/local/lib/x86_64-linux-gnu/perl/5.28.0/Tk/Widget.pm line 322
main::SelectFindMatch at XYZ.PL line 5958
main::mnuSRFindReplace at XYZ.PL line 4006


I knew, that a malformed UTF-8 character placed in source can cause problems. I eliminated memory problems by loading a file with 11181 lines inserting my search string at the very end and case insensitive search was Ok. Then I broke my source file into peaces and after several tests I found the wrongdoer. It was an Ohm sign character, perfectly shining in the editor window within a comment line. I make heavy use of unicode in comment lines, so everything seemed to be natural; I don't remember, how I placed this character into that line (my last edit there was in february 2017, I suppose it was a cut and paste action), but obviously it was not properly encoded (though file operations with this editor are demanding unicode encoding). Deleting this character solved the problem (Notice that the bug occured some 2000 lines above line 5958 and the subroutine containing it hasn't any relation to the one which raised the faulty behaviour).

It looks like the isutf8 tool and Perl (using UTF8 which is a loose version of UTF-8) slightly disagree about encodings. Since my editor worked at least for one year precisely, containing the above mentioned character in the source, the latest Perl update seems to come with a more strict understanding of utf8 encoding.

This is another example for coincidences. On the occasion of the operating system update breaking my perl installation, I did some tests after repairing it and bang: A new problem appeared. Initally it was natural to suspect the update causing this bug too, but I had to learn, that it was my fault...

[paxmark1]

That is getting strict.  A fail due to an "ohm" sign in #comments.  Glad you were able to debug.