zas/install_intelr_graphic_linux_firmware.sh

UrbanVampire · 2021-06-06T19:09:16Z

@Jiab77, I'm new to linux. But also i'm very lazy. Today i got 6 missing firmwares on one single system, so what should i do? Six time copy-paste the firmware name? No way! I'm too lazy for it. So, let's add a bit of automatization. This version of the script automatically detects missing firmware and installs it. BTW, it detects not only Intel but ANY of missing FWs. Of course, this script requres some optimisation - i'm new to linux - but it do the job. Tested on ubuntu and linux mint.

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing fw detection added by UrbanVampire <bgg@ngs.ru>, 2021

BLUE="\033[1;34m"
NC="\033[0m"
NL="\n"
TEMPFILE="$(tempfile)"

echo -e "${NL}${BLUE}Analyzing missing FWs, please wait...${NC}${NL}"

sudo update-initramfs -u 2>&1 >/dev/null | \
grep 'Possible missing firmware' | \
sed -n 's/^.*firmware//p'| sed -n 's/ for.*$//p' | \
while IFS= read -r line; \
do \
echo -e "${NL}${BLUE}Downloading missing driver $line...${NC}${NL}"; \
curl -o "$TEMPFILE" "https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain$line" || ex$
echo -e "${NL}${BLUE}Installing downloaded driver...${NC}${NL}" ; \
sudo mv -v "$TEMPFILE" "/lib/firmware$line" || exit 3 ; \
done

echo -e "${NL}${BLUE}Updating all kernels...${NC}${NL}"
read -p "Press enter to continue or CTRL+C to skip updating all kernels > "
sudo update-initramfs -u -k all

echo -e "${NL}${BLUE}Finished.${NC}${NL}"

Jiab77 · 2021-06-09T21:43:16Z

@UrbanVampire, That's an awesome idea you had! 🤘 I'll try it now.

I got recently the same problem but with nVidia related missing firmware files... shall we then try to expand this script to handle also this case?

apt/term.log:W: Possible missing firmware /lib/firmware/nvidia/465.24.02/gsp.bin for module nvidia
apt/term.log:W: Possible missing firmware /lib/firmware/nvidia/465.24.02/gsp.bin for module nvidia

You can check on your side if you have the same issues by run the following commands:

# Move to the main log folder
cd /var/log

# Search for the "missing" pattern
# Using "sudo" to be able to read "root" owned log files
sudo grep -R --color=always missing

Jiab77 · 2021-06-09T21:50:05Z

@UrbanVampire, I just tried your script version and it seems to have handled the case I've raised with the missing nVidia firmware but failed to move the downloaded file to the final destination:

Analyzing missing FWs, please wait...


Downloading missing driver /nvidia/465.24.02/gsp.bin...

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3187    0  3187    0     0   1053      0 --:--:--  0:00:03 --:--:--  1053

Installing downloaded driver...

mv: impossible de déplacer '/tmp/fileJx34mu' vers '/lib/firmware/nvidia/465.24.02/gsp.bin': Aucun fichier ou dossier de ce type

Updating all kernels...

Press enter to continue or CTRL+C to skip updating all kernels > ^C

I think this line has been cut by nano or similar text editor:

curl -o "$TEMPFILE" "https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain$line" || ex$

I guess it should be that instead:

curl -o "$TEMPFILE" "https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain$line" || exit 2 ; \

I'll try to debug the script and let you know if I find something or not 😉

Jiab77 · 2021-06-09T22:27:08Z

@UrbanVampire, I've been able to find out why it was failing that way:

Analyzing missing FWs, please wait...

Downloading missing driver /nvidia/465.24.02/gsp.bin...

URL: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/nvidia/465.24.02/gsp.bin

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3187    0  3187    0     0  32191      0 --:--:-- --:--:-- --:--:-- 32191

Error code: 1

Installing downloaded driver...

mv: impossible de déplacer '/tmp/filewJVEtF' vers '/lib/firmware/nvidia/465.24.02/gsp.bin': Aucun fichier ou dossier de ce type

Updating all kernels...

Press enter to continue or CTRL+C to skip updating all kernels > ^C

Here, the curl returned code test was invalid and it's why it returns 1 instead of 0 when corrected...

I've initially used [[ ! $? == 0 ]] instead of [[ $? == 0 ]] which then lead to the 1 returned instead of the proper curl return code or what is defined as exit value, 2.

So I've just added more debugging output and improved the coloring:

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing fw detection added by UrbanVampire <bgg@ngs.ru>, 2021

# Colors
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"

# Config
TEMPFILE="$(tempfile)"

echo -e "${NL}${BLUE}Analyzing missing FWs, please wait...${NC}${NL}"

sudo update-initramfs -u 2>&1 >/dev/null | \
grep 'Possible missing firmware' | \
sed -n 's/^.*firmware//p'| sed -n 's/ for.*$//p' | \
while IFS= read -r line; \
do \
echo -e "${BLUE}Downloading missing driver ${GREEN}${line}${BLUE}...${NC}${NL}" ; \
echo -e "${YELLOW}URL: ${WHITE}https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain$line${NC}${NL}" ; \
curl -o "$TEMPFILE" "https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain$line" || exit 2 ; \
if [[ $? == 0 ]]; \
then \
echo -e "${NL}${YELLOW}Error code: ${GREEN}${?}${NC}" ; \
else \
echo -e "${NL}${YELLOW}Error code: ${RED}${?}${NC}" ; \
fi ; \
echo -e "${NL}${BLUE}Installing downloaded driver...${NC}${NL}" ; \
sudo mv -v "$TEMPFILE" "/lib/firmware$line" || exit 3 ; \
done

echo -e "${NL}${BLUE}Updating all kernels...${NC}${NL}"
read -p "Press enter to continue or CTRL+C to skip updating all kernels > "
sudo update-initramfs -u -k all

echo -e "${NL}${BLUE}Finished.${NC}${NL}"

Then I've tested the generated URL: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/nvidia/465.24.02/gsp.bin and seen that the requested file was not existing under that location, which lead to an invalid temp file creation and then later failed when attempting to move the temp file to the final destination.

I've then tried to test this script variant:

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing fw detection added by UrbanVampire <bgg@ngs.ru>, 2021

# Colors
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"

# Config
TEMPFILE="$(tempfile)"

echo -e "${NL}${BLUE}Analyzing missing FWs, please wait...${NC}${NL}"

sudo update-initramfs -u 2>&1 >/dev/null | \
grep 'Possible missing firmware' | \
sed -n 's/^.*firmware//p'| sed -n 's/ for.*$//p' | \
while IFS= read -r line; \
do \
echo -e "${BLUE}Downloading missing driver ${GREEN}${line}${BLUE}...${NC}${NL}" ; \
echo -e "${YELLOW}URL: ${WHITE}https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain${line}${NC}${NL}" ; \
curl -sSL -o "$TEMPFILE" "https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain$line" || exit 2 ; \
if [[ $? == 0 ]]; \
then \
echo -e "${YELLOW}Error code: ${GREEN}${?}${NC}" ; \
echo -e "${NL}${BLUE}Installing downloaded driver...${NC}${NL}" ; \
sudo mv -v "$TEMPFILE" "/lib/firmware$line" || exit 3 ; \
else \
echo -e "${NL}${YELLOW}Error code: ${RED}${?}${NC}" ; \
echo -e "${NL}${RED}Could not find requested missing firmware.${NC}" ; \
fi ; \
done

echo -e "${NL}${BLUE}Updating all kernels...${NC}${NL}"
read -p "Press enter to continue or CTRL+C to skip updating all kernels > "
sudo update-initramfs -u -k all

echo -e "${NL}${BLUE}Finished.${NC}${NL}"

But as long as a 404 error does not alter the returned error code of curl, I have no idea so far how to handle "file not found" errors with curl without extensive HTML parsing or testing the returned HTTP error codes...

I did another test with curl but using -kIL instead of -sSL as arguments:

jiab77@[REDACTED]:~$ curl -kIL https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/nvidia/465.24.02/gsp.bin
HTTP/1.1 404 Not found
Server: nginx
Date: Wed, 09 Jun 2021 22:44:22 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Last-Modified: Wed, 09 Jun 2021 22:44:22 GMT
Expires: Thu, 01 Jan 1970 00:00:05 GMT

jiab77@[REDACTED]:~$ echo $?
0

and I think we could try to run this command first, parse the returned first line to detected if it returns 404 or 200 (to stay simple) and finally let the script continue if 200 is returned or stop when it's any other HTTP codes 🤘

What do you think?

UrbanVampire · 2021-06-10T03:51:17Z

@JiHab Maybe we should replace curl with wget? It returns errorcode 8 when serverside 404 is occurred.

Jiab77 · 2021-06-10T18:18:18Z

@UrbanVampire, that's a good idea! I was using wget initially but it has been changed to curl with the contribution of @mikkorantalainen. We can simply move back to wget and see if that solve the issue.

UrbanVampire · 2021-06-12T07:19:37Z

@Jiab77 First of all, the code became a bit hard to read and understand itself and produces too much output. So i'd suggest to (1) define a function and (2) suppress some outputs. R U OK with that? Just IMHO.
Second: I really don't know where to find nVidia FW's. AFAIK, it should be generated during proprietary driver installation process. I have no nV cards but I've heard that some versions of it's drvs and utls causes problems with FW's. But if I wrong and You know where to download nV FW's - well, let's add it to the code.
Anyway (this is third) we sure have to add some error handling.
And another thing. Some guys sets sudo password timeout to 0. It increases the system security but in our case it will lead to constant "enter sudo password" nightmare. I think we should check if user has su rights, and if not - ask to run the script with sudo.

UrbanVampire · 2021-06-12T20:21:43Z

@Jiab77, still dunno what to do with nVidia FWs. Look here, BTW, maybe it will help.

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing FW detection added by UrbanVampire <bgg@ngs.ru>, 2021
#
# Some definitions
# Colors:
DEBUG=0
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"
TAB="\t"
#
# Function to print an error message
function ErrorMessage(){
echo -e "${RED}ERROR${NC}"; [[ $DEBUG -eq 1 ]] && echo -e "${YELLOW}$OUTPUT${NC}${NL}"
}
#
# Function to process single missing FW
function ProcessFirmware(){
#
# Make sure that we have an parameter
if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}ProcessFirmware function called w/o parameters. Something went really wrong...${NC}${NL}"; return; fi
#
# Ok, first we try to download FW from git.kernel.org
echo -ne "${GREEN}$1:${TAB}${BLUE}Downloading... ${NC}"
local TEMPFILE="$(tempfile)"		# Get a temporary filename
local URL="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain"
OUTPUT=$(wget -nv -O "$TEMPFILE" $URL$1 2>&1)
if [[ $? -ne 0 ]]; then
	#
	#
	# Here we can add some code for other FW sources,
	# something like https://some-nvidia-site, for example.
	# But right now let's just report an error.
	#
	#
	ErrorMessage
	rm -f "$TEMPFILE"
	return
fi
#
# Now let's try to move downloaded file to /lib/firmware
echo -ne "${BLUE}Installing... ${NC}"
OUTPUT=$(mv -v "$TEMPFILE" "/lib/firmware$line" 2>&1)
if [[ $? -ne 0 ]]; then ErrorMessage; rm -f "$TEMPFILE"; return; fi
#
# Everything went Ok
echo -e "${GREEN}Ok${NC}"
}
#
#################################
#
# Here's the main body
#
# Let's make sure that we are superuser
if [[ $EUID -ne 0 ]]; then echo -e "${NL}${RED}Must be run with superuser privileges:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
#
# Is there some parameters?
if [[ $# -ne 0 ]] ; then
	if [[ $1 = "--debug" ]] ; then DEBUG=1; echo -e "${NL}${YELLOW}Debugging enabled.${NC}${NL}"
	else echo -e "${NL}${RED}Usage:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
else echo -e "${NL}${CYAN}Hint: ${BLUE}You can use the${WHITE} --debug ${BLUE}option for extended error info.${NC}${NL}"; fi
#
# Here we call update-initramfs, grep-search it's output for "missing HW" message
# then sed-extract every HW's filename and pass it to the ProcessFirmware function.
echo -e "${BLUE}Detecting missing FWs. It could take some time, please wait...${NC}${NL}"
update-initramfs -u 2>&1 >/dev/null | grep 'Possible missing firmware' | \
sed -n 's/^.*firmware//p'| sed -n 's/ for.*$//p' | \
while IFS= read -r line; do ProcessFirmware $line; done
#
# Now we need to re-generate all kernels
#echo -e "${NL}${BLUE}It's time to re-generate kernels. Press ${GREEN}Enter ${BLUE}to continue or ${RED}CTRL+C ${BLUE}to skip:${NC}"
#read
echo -e "${NL}${BLUE}Generating kernels. It could take some time, please wait...${NC}${NL}"
sudo update-initramfs -u -k all | grep 'Generating'
echo -e "${NL}${GREEN}Finished.${NC}${NL}"

Jiab77 · 2021-06-13T22:29:16Z

@UrbanVampire, what you did is just amazing! I'll try to find a solution for the nvidia part, maybe a simple find on my computer will tell me that the files are already existing but probably not at the expected place 😉

Otherwise, we just have to find if nVidia does having an open source driver repo or probably download the .run file, extract it and see if the missing firmware files does exist in the .run file.

Maybe it's probably too much work and we could simple filter nvidia errors?

Jiab77 · 2021-06-13T22:45:03Z

@UrbanVampire, there is an issue with the current script and it does not stop after the download error. I'm trying to fix it but so far everything I've tried is failing...

Jiab77 · 2021-06-13T23:17:13Z

@UrbanVampire, I've finally managed how to handle properly the returned error codes, here is the new version:

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing FW detection added by UrbanVampire <bgg@ngs.ru>, 2021
#
# Some definitions
# Config
DEBUG=0
# Colors:
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"
TAB="\t"
#
# Function to print an error message
function ErrorMessage(){
    echo -e "${RED}ERROR${NC}"; [[ $DEBUG -eq 1 ]] && echo -e "${NL}${YELLOW}$OUTPUT${NC}${NL}"
}
#
# Function to process single missing FW
function ProcessFirmware(){
    #
    # Make sure that we have an parameter
    if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}ProcessFirmware function called w/o parameters. Something went really wrong...${NC}${NL}"; return; fi
    #
    # Ok, first we try to download FW from git.kernel.org
    echo -ne "${GREEN}$1:${TAB}${BLUE}Downloading... ${NC}"
    local TEMPFILE="$(tempfile)"		# Get a temporary filename
    local URL="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain"
    OUTPUT=$(wget -nv -O "$TEMPFILE" $URL$1 2>&1 ; echo $? > /tmp/WGET_STATUS)
    if [[ $(cat /tmp/WGET_STATUS 2>/dev/null) -ne 0 ]]; then
        #
        #
        # Here we can add some code for other FW sources,
        # something like https://some-nvidia-site, for example.
        # But right now let's just report an error.
        #
        #
        ErrorMessage
        rm -f "$TEMPFILE"
        exit 2
    fi
    #
    # Now let's try to move downloaded file to /lib/firmware
    echo -ne "${BLUE}Installing... ${NC}"
    OUTPUT=$(mv -v "$TEMPFILE" "/lib/firmware$line" 2>&1 ; echo $? > /tmp/INSTALL_STATUS)
    if [[ $(cat /tmp/INSTALL_STATUS 2>/dev/null) -ne 0 ]]; then ErrorMessage; rm -f "$TEMPFILE"; exit 3; fi
    #
    # Everything went Ok
    echo -e "${GREEN}Ok${NC}"
}
#
#################################
#
# Here's the main body
#
# Let's make sure that we are superuser
if [[ $EUID -ne 0 ]]; then echo -e "${NL}${RED}Must be run with superuser privileges:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
#
# Is there some parameters?
if [[ $# -ne 0 ]] ; then
	if [[ $1 = "--debug" ]] ; then
        DEBUG=1; echo -e "${NL}${YELLOW}Debugging enabled.${NC}${NL}"
	else
        echo -e "${NL}${RED}Usage:${NC} sudo $0 [--debug]${NL}"; exit 1
    fi
else
    echo -e "${NL}${CYAN}Hint: ${WHITE}You can use the${GREEN} --debug ${WHITE}option for extended error info.${NC}${NL}"
fi
#
# Here we call update-initramfs, grep-search it's output for "missing HW" message
# then sed-extract every HW's filename and pass it to the ProcessFirmware function.
echo -e "${BLUE}Detecting missing FWs. It could take some time, please wait...${NC}${NL}"
update-initramfs -u 2>&1 >/dev/null | grep 'Possible missing firmware' | \
sed -n 's/^.*firmware//p'| sed -n 's/ for.*$//p' | \
while IFS= read -r line; do ProcessFirmware $line; done
#
# Now we need to re-generate all kernels
if [[ $1 == "--debug" ]] ; then
    echo -e ${WHITE}Error code: ${YELLOW}$(cat /tmp/WGET_STATUS 2>/dev/null)${NC}
fi
if [[ $(cat /tmp/WGET_STATUS 2>/dev/null) -eq 0 && $(cat /tmp/INSTALL_STATUS 2>/dev/null) -eq 0 ]]; then
    echo -e "${NL}${WHITE}It's time to re-generate kernels. Press ${GREEN}Enter ${WHITE}to continue or ${RED}CTRL+C ${WHITE}to skip:${NC}"
    read
    echo -e "${NL}${BLUE}Generating kernels. It could take some time, please wait...${NC}${NL}"
    sudo update-initramfs -u -k all | grep 'Generating'
else
    if [[ $(cat /tmp/WGET_STATUS 2>/dev/null) -ne 0 ]]; then
        echo -e "${NL}${RED}Error: ${WHITE}Cound not download the missing firmware.${NC}${NL}"
    fi
    if [[ $(cat /tmp/INSTALL_STATUS 2>/dev/null) -ne 0 ]]; then
        echo -e "${NL}${RED}Error: ${WHITE}Unexpected error during firmware install.${NC}${NL}"
    fi
fi
rm -f /tmp/{WGET_STATUS,INSTALL_STATUS} 2>/dev/null
echo -e "${GREEN}Finished.${NC}${NL}"

I'm still having to see if the missing firmware already exist or not on my computer. I'll let you know.

I've also tuned a little the text colors, let me know if you like it or not.

Jiab77 · 2021-06-13T23:23:10Z

@UrbanVampire, I've been able to find a way to get the missing firmware file but I'm not sure that we should handle it and automate the whole process...

Go to https://www.nvidia.com/en-us/drivers/unix/
Get the correct version: https://us.download.nvidia.com/XFree86/Linux-x86_64/465.31/NVIDIA-Linux-x86_64-465.31.run
Lookup for the missing firmware in the downloaded file: sh NVIDIA-Linux-x86_64-465.31.run --list | grep -i gsp.bin
Extract the archive: sh NVIDIA-Linux-x86_64-465.31.run -x
Create missing firmware directory: sudo mkdir -pv /lib/firmware/nvidia/465.24.02/
Copy extracted firmware to local nvidia firmware directory: sudo cp -v NVIDIA-Linux-x86_64-465.31/firmware/gsp.bin /lib/firmware/nvidia/465.24.02/
Update current kernel to see if the missing firmware error still exist: sudo update-initramfs -u
Congrats, no more missing firmware error 🎉 🍻
(optional) Update all kernels: sudo update-initramfs -u -k all

I forgot to mention that you can get the available options of the nVidia archive with:

sh NVIDIA-Linux-x86_64-465.31.run --help
sh NVIDIA-Linux-x86_64-465.31.run --advanced-options

Jiab77 · 2021-06-13T23:40:28Z

@Jiab77 First of all, the code became a bit hard to read and understand itself and produces too much output. So i'd suggest to (1) define a function and (2) suppress some outputs. R U OK with that? Just IMHO.
Second: I really don't know where to find nVidia FW's. AFAIK, it should be generated during proprietary driver installation process. I have no nV cards but I've heard that some versions of it's drvs and utls causes problems with FW's. But if I wrong and You know where to download nV FW's - well, let's add it to the code.
Anyway (this is third) we sure have to add some error handling.
And another thing. Some guys sets sudo password timeout to 0. It increases the system security but in our case it will lead to constant "enter sudo password" nightmare. I think we should check if user has su rights, and if not - ask to run the script with sudo.

@UrbanVampire, I'm agree with you that's why I've tried to improve a little the script as I'm having an nvidia card on my side 😉 I hope you'll like the changes but as said previously, handling the whole download + search + copy if found automatically from the script seems to me a little overkill, maybe we can simply direct the user to the nvidia download page like:

# if the string 'nvidia' is detected
echo -e "${RED}Error: Unable to find missing firmware in kernel repo, please navigate to ${WHITE}https://www.nvidia.com/en-us/drivers/unix/${WHITE} ${RED}and run the following commands: blah blah blah${NC}${NL}"

Something like that, what do you think?

UrbanVampire · 2021-06-14T17:11:04Z

@Jiab77 Today i've tested the script on Debian 10. Well, it works. But we got another issue,
Possible missing firmware /lib/firmware/rtl_nic/rtl8168e-1.fw for module r8169
Files are downloaded Ok, but /lib/firmware/rtl_nic/ folder doesn't exist (there is /lib/firmware/ but there is NO rtl_nic/ in it), so mv fails with error.
We have to do something here:
OUTPUT=$(mv -v "$TEMPFILE" "/lib/firmware$line" 2>&1 ; echo $? > /tmp/INSTALL_STATUS)
Do You know the easy way to fix it?

UrbanVampire · 2021-06-14T17:15:31Z

@UrbanVampire, there is an issue with the current script and it does not stop after the download error. I'm trying to fix it but so far everything I've tried is failing...

This is not an issue, this is feature :)
If the script fails with downloading one file (for nVidia for example), why sould we give up with another one? Or another 18. I'm not joking, today i've seen 19 missing firmwares on one system.
So I think we don't need to stop at any error but just report it and try to continue.

UrbanVampire · 2021-06-14T17:45:06Z

And one more thing. I'm not a big fan of storing data in temporary files. I'd suggest to add a couple of variables to count download an installation errors instead of writing exitcodes to tempfiles. If they are zero before kernel regeneration - we're ok to go, if not - ask user what to do next.
And no, I don't think we should give up with nVidia. Let's automate it.

UrbanVampire · 2021-06-15T09:07:54Z

I get it.... It's a child process, so variables do not return it's values.
Well... I found a workaround :)

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing FW detection added by UrbanVampire <bgg@ngs.ru>, 2021
#
# Some definitions
# Config
DEBUG=0     # Debug flag
FWTotal=0   # Total missing FWs counter
FWSuccs=0   # Succesful installed FWs counter
FWError=0   # Failed FWs
NVDwnld=0   # 
# Colors:
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"
TAB="\t"
#
# Function to execute a comand and get it's output
function DebuggedExec(){
	#
	# Make sure that we have an parameter
	if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}DebuggedExec function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
        #
        OUTPUT=$(eval $1 2>&1)			# Let's execute the given command
	if [[ $? -ne 0 ]]; then			# Do we have an error?
		((FWError++))			# Yes, we have.
		echo -e "${RED}Failed${NC}"
		if [[ $DEBUG -eq 1 ]]; then	# Are we in debugging mode?
			echo -e "${RED}Executed command was:${TAB}${YELLOW}$1${NC}"
			echo -e "${RED}Error message was:${TAB}${YELLOW}$OUTPUT${NC}"
		fi
		return 1
	fi
	return 0			# No errors, everything went fine
}
#
# Function to process single missing FW
function ProcessFirmware(){
	#
	# Make sure that we have an parameter
	if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}ProcessFirmware function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
	#
	((FWTotal++))		# We got a missing FW, let's count it.
	#
	# Let's extract FW name and full path:
	FWFullPath=$(echo $1 | sed -n 's/^.*firmware //p'| sed -n 's/ for.*$//p')
	FWFileName=$(echo $1 | sed -n 's/^.*firmware//p'| sed -n 's/ for.*$//p')
	echo -ne "${GREEN}$FWFileName${BLUE}:${TAB}Downloading... ${NC}"
	#
	# Is it nVidia?
	if [[ "$FWFileName" == *"nvidia"* ]]; then
#	if [[ "$FWFileName" == *"rtl8168g-2"* ]]; then
#	if [[ "$FWFileName" == *"rtl_nic"* ]]; then
		echo -e "${CYAN}nVidia FW detected. Skipping for now.${NC}"
	else # No, this is not nVidia, let's try to download FW from git.kernel.org
	        local TEMPFILE="$(tempfile)"		# Get a temporary filename
        	local URL="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain"
        	DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL$FWFileName"
		if [[ $? -ne 0 ]]; then return 1; fi
		# Now let's try to move downloaded file to /lib/firmware
		echo -ne "${BLUE}Installing... ${NC}"
		# First we need to check if path is exists
        	DebuggedExec "mkdir -p ${FWFullPath%/*}"
		[[ $? -eq 0 ]] || return 1
		# Moving the file to it's place
        	DebuggedExec "mv -v \"$TEMPFILE\" $FWFullPath"
		[[ $? -eq 0 ]] || return 1
		#
		# Everything went Ok
		((FWSuccs++))
		echo -e "${GREEN}Ok${NC}"
		return 0
		echo "Total: $FWTotal, Succes $FWSuccs"
    	fi
}
#
#################################
#
# Here's the main body
#
# Let's make sure that we are superuser
if [[ $EUID -ne 0 ]]; then echo -e "${NL}${RED}Must be run with superuser privileges:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
#
# Is there some parameters?
if [[ $# -ne 0 ]] ; then
	if [[ $1 = "--debug" ]] ; then
		DEBUG=1; echo -e "${NL}${YELLOW}Debugging enabled.${NC}${NL}"
	else
		echo -e "${NL}${RED}Usage:${NC} sudo $0 [--debug]${NL}"; exit 1
	fi
else
	echo -e "${NL}${CYAN}Hint: ${WHITE}You can use the${GREEN} --debug ${WHITE}option for extended error info.${NC}${NL}"
fi
#
# Here we call update-initramfs, grep-search it's output for "missing HW" message
echo -e "${BLUE}Detecting missing FWs. It could take some time, please wait...${NC}${NL}"
MFWs=$(update-initramfs -u 2>&1 >/dev/null | grep 'Possible missing firmware')
# Let's process missing FWs one by one
while IFS= read -r line; do ProcessFirmware "$line"; done < <(echo "$MFWs")
#
# Did we found some missing FWs?
if [[ FWTotal -eq 0 ]]; then echo -e "$${BLUE}No missing FWs found. Nothing to do. Exiting...${NC}${NL}"; exit 0; fi
# Is there some successful FWs?
if [[ FWSuccs -eq 0 ]]; then		# Nope, no luck
	echo -ne "${NL}${YELLOW}WARNING: No FWs found or downloaded. See messages above"
	[[ $DEBUG -eq 0 ]] && echo -ne " or try --debug option for more info"
	echo -ne ". Exiting...${NC}${NL}"
	exit 1
fi
# Maybe ALL FWs downloaded with success?
if [[ FWSuccs -ne FWTotal ]]; then	# Nope
	echo -ne "${NL}${YELLOW}WARNING: Some FWs was not found or not downloaded. See messages above"
	[[ $DEBUG -eq 1 ]] || echo -ne " or try --debug option for more info"
	echo -ne ". But You still can regenerate kernels.${NC}${NL}"
fi
# Now we need to re-generate all kernels
echo -ne "${NL}${BLUE}It's time to re-generate kernels. Press ${GREEN}Enter ${BLUE}to continue or ${RED}CTRL+C ${BLUE}to skip:${NC}"
read
echo -e "${NL}${BLUE}Generating kernels. It could take some time, please wait...${NC}${NL}"
sudo update-initramfs -u -k all | grep 'Generating'
echo -e "${NL}${GREEN}Finished.${NC}${NL}"

Now i'm working on nvidia FWs, pltase be patient.

UrbanVampire · 2021-06-15T12:36:10Z

@Jiab77 You know... I think we did it :)

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 / 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing FW detection added by UrbanVampire <bgg@ngs.ru>, 2021
#
# Some definitions
# Config
DEBUG=0     # Debug flag
FWTotal=0   # Total missing FWs counter
FWSuccs=0   # Succesful installed FWs counter
FWError=0   # Failed FWs
declare -a InstalledFWs	# Array to store and check alredy installed FWs
# Colors:
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"
TAB="\t"
#
# Function to execute a comand and get it's output
function DebuggedExec(){
	#
	# Make sure that we have an parameter
	if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}DebuggedExec function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
        #
        OUTPUT=$(eval $1 2>&1)			# Let's execute the given command
	if [[ $? -ne 0 ]]; then			# Do we have an error?
		((FWError++))			# Yes, we have.
		echo -e "${RED}Failed${NC}"
		if [[ $DEBUG -eq 1 ]]; then	# Are we in debugging mode?
			echo -e "${RED}Executed command was:${TAB}${YELLOW}$1${NC}"
			echo -e "${RED}Error message was:${TAB}${YELLOW}$OUTPUT${NC}"
		fi
		return 1
	fi
	return 0			# No errors, everything went fine
}
#
# Function to process single missing FW
function ProcessFirmware(){
	#
	# Make sure that we have an parameter
	if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}ProcessFirmware function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
	#
	((FWTotal++))		# We got a missing FW, let's count it.
	#
	# Let's extract FW name and full path:
	FWFullPath=$(echo $1 | sed -n 's/^.*firmware //p')
	FWFileName=$(echo $1 | sed -n 's/^.*firmware//p')
	#
	if [[ $FWTotal -eq 1 ]]; then	# Is it first line? If so let's calculate tabulation offset
		let TABoff=$MaxLenght-${#1}+${#FWFileName}+2
	fi
	echo -ne "${GREEN}$FWFileName${BLUE}:${TAB}${NC}\033[50D\033[${TABoff}C"
	#
	# Time to check if this file alredy installed
	if [[ " ${InstalledFWs[*]} " == *"$FWFileName"* ]]; then
		((FWSuccs++))			# Everything went Ok
		echo -e "${CYAN}Alredy installed, Skipping${NC}"
		return 0
	fi	
	echo -ne "${BLUE}Downloading... ${NC}"
        local TEMPFILE="$(tempfile)"		# Get a temporary filename
	#
	# Is it nVidia?
	if [[ "$FWFileName" == *"nvidia"* ]]; then
		echo -ne "${CYAN}nVidia FW detected. It could take some time.${NL}\033[50D\033[${TABoff}CPlease wait... ${NC}"
		NVVersion=$(echo $FWFileName | sed -n 's/^.*nvidia\///p'| sed -n 's/\/.*$//p') # Extract version
		FolderNam="NVIDIA-Linux-x86_64-$NVVersion"
		RunFlName="$FolderNam.run"
		NVFWFlNme=$(echo $FWFileName | sed -n 's/^.*\///p') # Extract nV FW filename
		URL="https://download.nvidia.com/XFree86/Linux-x86_64/$NVVersion/$RunFlName"
        	DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL"
		if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
		echo -ne "${BLUE}Extracting... ${NC}"
        	DebuggedExec "sh \"$TEMPFILE\" -x --target /tmp/$FolderNam"
		if [[ $? -ne 0 ]]; then rm "$TEMPFILE"; rm -drf /tmp/$FolderNam ; return 1; fi
		rm "$TEMPFILE"
        	DebuggedExec "mv /tmp/$FolderNam/firmware/$NVFWFlNme \"$TEMPFILE\""
		if [[ $? -ne 0 ]]; then rm -drf /tmp/$FolderNam ; return 1; fi
		# file is extracted and ready to go
		rm -drf /tmp/$FolderNam			# Some cleanup
	else # No, this is not nVidia, let's try to download FW from git.kernel.org
        	local URL="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain"
        	DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL$FWFileName"
		if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    	fi
	# Now let's try to move downloaded file to /lib/firmware
	echo -ne "${BLUE}Installing... ${NC}"
	# First we need to check if path is exists
       	DebuggedExec "mkdir -p ${FWFullPath%/*}"
	if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
	# Moving the file to it's place
       	DebuggedExec "mv -v \"$TEMPFILE\" $FWFullPath"
	if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
	#
	((FWSuccs++))				# Everything went Ok
	echo -e "${GREEN}Ok${NC}"
	InstalledFWs[$FWSuccs]=$FWFileName	# Store FW in list of installed FWs
	return 0
}
#
#################################
#
# Here's the main body
#
# Let's make sure that we are superuser
if [[ $EUID -ne 0 ]]; then echo -e "${NL}${RED}Must be run with superuser privileges:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
#
# Is there some parameters?
if [[ $# -ne 0 ]] ; then
	if [[ $1 = "--debug" ]] ; then
		DEBUG=1; echo -e "${NL}${YELLOW}Debugging enabled.${NC}${NL}"
	else
		echo -e "${NL}${RED}Usage:${NC} sudo $0 [--debug]${NL}"; exit 1
	fi
else
	echo -e "${NL}${CYAN}Hint: ${WHITE}You can use the${GREEN} --debug ${WHITE}option for extended error info.${NC}${NL}"
fi
#
# Here we call update-initramfs, grep-search it's output for "missing HW" message
echo -e "${BLUE}Detecting missing FWs. It could take some time, please wait...${NC}${NL}"
MFWs=$(update-initramfs -u 2>&1 >/dev/null | grep 'Possible missing firmware' | sed -n 's/ for.*$//p') 
#
MaxLenght=0	# Get longest string lenght - we'll need it later to calculate tabulation
while IFS= read -r line; do if [[ ${#line} -gt $MaxLenght ]]; then MaxLenght=${#line}; fi; done < <(echo "$MFWs")
#
# Let's process missing FWs one by one
while IFS= read -r line; do ProcessFirmware "$line"; done < <(echo "$MFWs")
#
# Did we found some missing FWs?
if [[ $FWTotal -eq 0 ]]; then echo -e "$${BLUE}No missing FWs found. Nothing to do. Exiting...${NC}${NL}"; exit 0; fi
# Is there some successful FWs?
if [[ $FWSuccs -eq 0 ]]; then		# Nope, no luck
	echo -ne "${NL}${YELLOW}WARNING: No FWs found or downloaded. See messages above"
	[[ $DEBUG -eq 0 ]] && echo -ne " or try --debug option for more info"
	echo -ne ". Exiting...${NC}${NL}"
	exit 1
fi
# Maybe ALL FWs downloaded with success?
if [[ $FWSuccs -ne $FWTotal ]]; then	# Nope
	echo -ne "${NL}${YELLOW}WARNING: Some FWs was not found or not downloaded. See messages above"
	[[ $DEBUG -eq 1 ]] || echo -ne " or try --debug option for more info"
	echo -ne ". But You still can regenerate kernels.${NC}${NL}"
fi
# Now we need to re-generate all kernels
echo -ne "${NL}${BLUE}It's time to re-generate kernels. Press ${GREEN}Enter ${BLUE}to continue or ${RED}CTRL+C ${BLUE}to skip:${NC}"
read
echo -e "${NL}${BLUE}Generating kernels. It could take some time, please wait...${NC}${NL}"
sudo update-initramfs -u -k all | grep 'Generating'
echo -e "${NL}${GREEN}Finished.${NC}${NL}"

There's one thing that bothers me. On nVidia download site there's different installers for 32 and 64 architecture. Do we have to check architecture or it doesn't matter for firmware?

UrbanVampire · 2021-06-15T12:47:10Z

@Jiab77, Also maybe we to add instruction for nVidia and display it if nV installations fails for some reason.
And - yes, forgot to mention: script tries to continue even if some files failed. I think it's right.

Jiab77 · 2021-06-16T23:45:00Z

@UrbanVampire, there is an issue with the current script and it does not stop after the download error. I'm trying to fix it but so far everything I've tried is failing...

This is not an issue, this is feature :)
If the script fails with downloading one file (for nVidia for example), why sould we give up with another one? Or another 18. I'm not joking, today i've seen 19 missing firmwares on one system.
So I think we don't need to stop at any error but just report it and try to continue.

hahaha good one "this is not an issue, this is a feature" 🤣
but I see what you mean and you are definitely right!

Jiab77 · 2021-06-16T23:53:13Z

And one more thing. I'm not a big fan of storing data in temporary files. I'd suggest to add a couple of variables to count download an installation errors instead of writing exitcodes to tempfiles. If they are zero before kernel regeneration - we're ok to go, if not - ask user what to do next.
And no, I don't think we should give up with nVidia. Let's automate it.

I got you, I'm not a fan of using temporary files neither, I see them as a temporary workaround until a better solution is found 😆 and in this case, as you understood in your next comment, it was because when running as child process, the variable does not hold the right value and always returned me 0 instead of the expected value from $? but when moved to temporary file, it worked because the expected value was stored correctly and then could be reused cat.

I know that was not the best solution but as I could see in the next version you've managed it very well by using internal functions! 🙇

Oh, I forgot to mention, holy fucking brilliant impressive work for the nVidia part 😮 🙌

Jiab77 · 2021-06-17T00:08:37Z

@Jiab77 You know... I think we did it :)

No, you did it 😁

There's one thing that bothers me. On nVidia download site there's different installers for 32 and 64 architecture. Do we have to check architecture or it doesn't matter for firmware?

@UrbanVampire, yes we have to detect it as some Linux distribution have a different lib folders when it is 32 bit or 64 bit like /lib and /lib64. I got both on my system for example but the lib64 folder is almost empty and just contains a symlink to a file stored in /lib 😅 but I'm using an Ubuntu based Linux distrib so maybe it can be different on original Ubuntu or any other Linux distribs.

What I can do on my side is to take a look at work if the directory structure stays similar on the different Linux distribs we have and if that's the case and that not matter the architecture, everything got installed into /lib/firmware then we just have to detect the architecture only to download to right binary file from the nVidia site.

Jiab77 · 2021-06-17T00:53:34Z

@UrbanVampire, as you mainly did the whole magic, I was wondering to first, ask you to make a new project on github and host the code of this script for better maintenance and probably access, to others willing to improve it. what do you think about that?

Here is a slightly modified version just to keep track of everything you've added in the script:

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 <jiab77@pm.me>, 2020
# Improved by Mikko Rantalainen <mikko.rantalainen@iki.fi>, 2020
# Automatic missing FW detection added by UrbanVampire <bgg@ngs.ru>, 2021
# Improved script colors and suggested handling missing nVidia firmware files by Jiab77 <jiab77@pm.me>, 2021
# Automagical missing nVidia firmware installation added by UrbanVampire <bgg@ngs.ru>, 2021
# Minor spellchecking and fixing, minor display fixes and credits added by Jiab77 <jiab77@pm.me>, 2021
#
# Some definitions
# Config
DEBUG=0     # Debug flag
FWTotal=0   # Total missing FWs counter
FWSuccs=0   # Succesful installed FWs counter
FWError=0   # Failed FWs
declare -a InstalledFWs	# Array to store and check alredy installed FWs
# Colors:
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"
TAB="\t"
#
# Function to execute a comand and get it's output
function DebuggedExec(){
    #
    # Make sure that we have an parameter
    if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}DebuggedExec function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
        #
        OUTPUT=$(eval $1 2>&1)			# Let's execute the given command
    if [[ $? -ne 0 ]]; then			# Do we have an error?
        ((FWError++))			# Yes, we have.
        echo -e "${RED}Failed${NC}"
        if [[ $DEBUG -eq 1 ]]; then	# Are we in debugging mode?
            echo -e "${RED}Executed command was:${TAB}${YELLOW}$1${NC}"
            echo -e "${RED}Error message was:${TAB}${YELLOW}$OUTPUT${NC}"
        fi
        return 1
    fi
    return 0			# No errors, everything went fine
}
#
# Function to process single missing FW
function ProcessFirmware(){
    #
    # Make sure that we have an parameter
    if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}ProcessFirmware function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
    #
    ((FWTotal++))		# We got a missing FW, let's count it.
    #
    # Let's extract FW name and full path:
    FWFullPath=$(echo $1 | sed -n 's/^.*firmware //p')
    FWFileName=$(echo $1 | sed -n 's/^.*firmware//p')
    #
    if [[ $FWTotal -eq 1 ]]; then	# Is it first line? If so let's calculate tabulation offset
        let TABoff=$MaxLength-${#1}+${#FWFileName}+2
    fi
    #
    # Time to check if this file alredy installed
    if [[ " ${InstalledFWs[*]} " == *"$FWFileName"* ]]; then
        ((FWSuccs++))			# Everything went Ok
        echo -e "${CYAN}Alredy installed, Skipping${NC}"
        return 0
    else
        # Moved this line here to avoid printing blank '$FWFileName' value when already installed
        # And then output something like ": Alredy installed, Skipping", with an extra ": "
        # Because '$FWFileName' is not set as nothing is missing.
        #
        # Maybe we should move the text "Alredy installed, Skipping" somewhere else?
        echo -ne "${GREEN}$FWFileName${BLUE}:${TAB}${NC}\033[50D\033[${TABoff}C"
    fi	
    echo -ne "${BLUE}Downloading... ${NC}"
    local TEMPFILE="$(tempfile)"		# Get a temporary filename
    #
    # Is it nVidia?
    if [[ "$FWFileName" == *"nvidia"* ]]; then
        echo -ne "${CYAN}nVidia FW detected. It could take some time.${NL}\033[50D\033[${TABoff}CPlease wait... ${NC}"
        NVVersion=$(echo $FWFileName | sed -n 's/^.*nvidia\///p'| sed -n 's/\/.*$//p') # Extract version
        FolderNam="NVIDIA-Linux-x86_64-$NVVersion"
        RunFlName="$FolderNam.run"
        NVFWFlNme=$(echo $FWFileName | sed -n 's/^.*\///p') # Extract nV FW filename
        URL="https://download.nvidia.com/XFree86/Linux-x86_64/$NVVersion/$RunFlName"
        DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL"
        if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
        echo -ne "${BLUE}Extracting... ${NC}"
        DebuggedExec "sh \"$TEMPFILE\" -x --target /tmp/$FolderNam"
        if [[ $? -ne 0 ]]; then rm "$TEMPFILE"; rm -drf /tmp/$FolderNam ; return 1; fi
        rm "$TEMPFILE"
        DebuggedExec "mv /tmp/$FolderNam/firmware/$NVFWFlNme \"$TEMPFILE\""
        if [[ $? -ne 0 ]]; then rm -drf /tmp/$FolderNam ; return 1; fi
        # file is extracted and ready to go
        rm -drf /tmp/$FolderNam			# Some cleanup
    else # No, this is not nVidia, let's try to download FW from git.kernel.org
        local URL="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain"
        DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL$FWFileName"
        if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    fi
    # Now let's try to move downloaded file to /lib/firmware
    echo -ne "${BLUE}Installing... ${NC}"
    # First we need to check if path is exists
    DebuggedExec "mkdir -p ${FWFullPath%/*}"
    if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    # Moving the file to it's place
    DebuggedExec "mv -v \"$TEMPFILE\" $FWFullPath"
    if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    #
    ((FWSuccs++))				# Everything went Ok
    echo -e "${GREEN}Ok${NC}"
    InstalledFWs[$FWSuccs]=$FWFileName	# Store FW in list of installed FWs
    return 0
}
#
#################################
#
# Here's the main body
#
# Let's make sure that we are superuser
if [[ $EUID -ne 0 ]]; then echo -e "${NL}${RED}Must be run with superuser privileges:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
#
# Is there some parameters?
if [[ $# -ne 0 ]] ; then
    if [[ $1 = "--debug" ]] ; then
        DEBUG=1; echo -e "${NL}${YELLOW}Debugging enabled.${NC}${NL}"
    else
        echo -e "${NL}${RED}Usage:${NC} sudo $0 [--debug]${NL}"; exit 1
    fi
else
    echo -e "${NL}${CYAN}Hint: ${WHITE}You can use the${GREEN} --debug ${WHITE}option for extended error info.${NC}${NL}"
fi
#
# Here we call update-initramfs, grep-search it's output for "missing HW" message
echo -e "${BLUE}Detecting missing FWs. It could take some time, please wait...${NC}${NL}"
MFWs=$(update-initramfs -u 2>&1 >/dev/null | grep 'Possible missing firmware' | sed -n 's/ for.*$//p') 
#
MaxLength=0	# Get longest string length - we'll need it later to calculate tabulation
while IFS= read -r line; do if [[ ${#line} -gt $MaxLength ]]; then MaxLength=${#line}; fi; done < <(echo "$MFWs")
#
# Let's process missing FWs one by one
while IFS= read -r line; do ProcessFirmware "$line"; done < <(echo "$MFWs")
#
# Did we found some missing FWs?
if [[ $FWTotal -eq 0 ]]; then echo -e "$${BLUE}No missing FWs found. Nothing to do. Exiting...${NC}${NL}"; exit 0; fi
# Is there some successful FWs?
if [[ $FWSuccs -eq 0 ]]; then		# Nope, no luck
    echo -ne "${NL}${YELLOW}WARNING: No FWs found or downloaded. See messages above"
    [[ $DEBUG -eq 0 ]] && echo -ne " or try --debug option for more info"
    echo -ne ". Exiting...${NC}${NL}"
    exit 1
fi
# Maybe ALL FWs downloaded with success?
if [[ $FWSuccs -ne $FWTotal ]]; then	# Nope
    echo -ne "${NL}${YELLOW}WARNING: Some FWs was not found or not downloaded. See messages above"
    [[ $DEBUG -eq 1 ]] || echo -ne " or try --debug option for more info"
    echo -ne ". But You still can regenerate kernels.${NC}${NL}"
fi
# Now we need to re-generate all kernels
echo -ne "${NL}${BLUE}It's time to re-generate kernels. Press ${GREEN}Enter ${BLUE}to continue or ${RED}CTRL+C ${BLUE}to skip:${NC}"
read
echo -e "${NL}${BLUE}Generating kernels. It could take some time, please wait...${NC}${NL}"
sudo update-initramfs -u -k all | grep 'Generating'
echo -e "${NL}${GREEN}Finished.${NC}${NL}"

I'm also wondering if we should move to case / esac instead of multiplying if [[ "$FWFileName" == *"firmware"* ]]; then to manage many other missing firmware as you did for nVidia, like for your missing rtl* firmware files?

This way we could manage several missing ones like that:

case "$FWFileName" in
*"nvidia"*)
    # code block
;;
*"rtl"*)
    # code block
;;
*)
    # code block for all other not supported cases, like for displaying a message
    # if all previous tests failed.
;;
esac

I don't have your bash scripting skills so I'm not really sure that is possible anyway 😅
oh, and for someone that claims to be a Linux beginner, you're just impressive! I'm feeling like knowing nothing 😆

Jiab77 · 2021-06-17T01:33:47Z

@UrbanVampire, about the architecture detection we could try something simple like this:

if [[ $(lscpu | grep -i x86_64 | wc -l) -eq 1 ]]; then
    # This architecture is 64 bit
else
    # This architecture is 32 bit
fi

I was pretty sure that I could get this kind information from the file /proc/cpuinfo also but I could not find any relevant and precise values that we could use to detect the architecture better. I've also looked at some other files in /proc but could not anything else that could do the job so maybe lscpu is the simplest way.

I can try to run strace lscpu and see if it query some files that we could use instead of calling lscpu directly.

Ok I've tried with strace lscpu 2>&1 | grep -vi "No such" | grep -i "openat" and the file /proc/cpuinfo is used but also many other ones:

$ strace lscpu 2>&1 | grep -vi "No such" | grep -i "openat"
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libsmartcols.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/cpuinfo", O_RDONLY) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/kernel_max", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/possible", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/present", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/thread_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/core_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/core_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/physical_package_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq", O_RDONLY|O_CLOEXEC) = 3
... (removed to reduce redondant lines due to multi cores)
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/thread_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/core_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/core_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/physical_package_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index0/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index1/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index3/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cpufreq/cpuinfo_max_freq", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cpufreq/cpuinfo_min_freq", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/node", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/usr/share/locale-langpack/fr/LC_MESSAGES/util-linux.mo", O_RDONLY) = 4
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/devices/system/node/node0/cpumap", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/sys/kernel/osrelease", O_RDONLY) = 3
openat(AT_FDCWD, "/sys/firmware/dmi/tables/DMI", O_RDONLY) = -1 EACCES (Permission denied)
openat(AT_FDCWD, "/proc/bus/pci/devices", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/bus/pci/devices", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/bus/pci/devices", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/self/status", O_RDONLY) = 3

Yes, I'm having a cpu with 8 cores, that's why the list of open files goes from 0 to 7 😅

UrbanVampire · 2021-06-23T03:53:53Z

@Jiab77, Sorry for late answer, it was a hard week on work.

No, you did it

Oh, c'mon, it was you who found nvidia workaround. Not to mention that I wouldn't even start working without your original script.

I got both on my system for example but the lib64 folder is almost empty and just contains a symlink to a file stored in /lib sweat_smile but I'm using an Ubuntu based Linux distrib so maybe it can be different on original Ubuntu or any other Linux distribs.

AFAIK, the "firmware" folder is placed in "/lib" on both 32 and 64 systems. But after recent patches "/lib" is moved to "/usr/lib" on some systems. So, maybe it's better to extract destination path from original "Possible missing hardware" message.

maybe lscpu is the simplest way.

I think the arch command is the simplest way. But there's other thing: latest 32-bit drivers on nvidia site is 390.143. I'm not really sure what FW version is needed for 32-bit kernels. Anyway we need some experiments to understand:

Are the firmware files the same in 32 and 64 nvidia driver versions?
Will 64-bit nvidia 'run' script unpack files on a 32-bit system?
Can nvidia 'run' script be unpacked with a regular untar?

I'm also wondering if we should move to case / esac instead of multiplying if

Now we have only 2 options, nvidia and NOT nvidia,, but i'll think about it.

ask you to make a new project on github

I have zero experience with github but i'll try :)

UrbanVampire · 2021-06-23T06:08:28Z

@Jiab77
https://github.com/UrbanVampire/missing-firmware-fix
Just fixed false "Already installed" for now.

UrbanVampire · 2021-06-24T10:44:24Z

@Jiab77

1. Are the firmware files the same in 32 and 64 nvidia driver versions?
2. Will 64-bit nvidia 'run' script unpack files on a 32-bit system?
3. Can nvidia 'run' script be unpacked with a regular untar?

Well, answer to "2" and "3" is "no". At least I haven't found a way to extract files from the installer using standard archivers. And the 64-bit installer uses 64-bit binary stub, so it won't run on a 32-bit systems. It means that "1" is doesn't matter cos' we cannot use 64 installer on 32 system.
The rest is easy: just download installer from "Linux-x86" or "Linux-x86_64" folder of nVidia site according to system architecture.
The code is updated.
I think for now my job is done here.
BTW, don't You wanna add German of French README translation? Sorry, don't know what Your native language is.

Jiab77 · 2021-06-28T22:56:35Z

@Jiab77, Sorry for late answer, it was a hard week on work.

@UrbanVampire, No worries, it's pretty same on my side, I'm also overloaded of work... that's why I'm replying you just now 😅

Oh, c'mon, it was you who found nvidia workaround. Not to mention that I wouldn't even start working without your original script.

Well, it was pretty simple to be honest but do the job to automate the whole process as you did is not the same at all!

AFAIK, the "firmware" folder is placed in "/lib" on both 32 and 64 systems. But after recent patches "/lib" is moved to "/usr/lib" on some systems. So, maybe it's better to extract destination path from original "Possible missing hardware" message.

Yes you're right and I saw in the code that you managed it very well. 🤘

I think the arch command is the simplest way. But there's other thing: latest 32-bit drivers on nvidia site is 390.143. I'm not really sure what FW version is needed for 32-bit kernels. Anyway we need some experiments to understand:

Thanks a lot for that, I was not aware about the arch command and it's much better than parsing lscpu output 😁

I have zero experience with github but i'll try :)

You did it very well for the less I could see 👍 I've stared the project already and added on my watch list 😜

I think for now my job is done here.
BTW, don't You wanna add German of French README translation? Sorry, don't know what Your native language is.

Well, I'm a native French speaker but I'm not really sure that would be useful to translate it in French honestly as many French native speakers that work in the computer domain knows reading English already and if they don't then they should learn it, it will give them a much wider source of knowledge 😜

BTW, let's keep in touch somewhere else, what do you think? write me an email if interested 😉

ErnestStCharly · 2021-07-22T15:43:17Z

Thanks a lot guys. Great script!!!

coitus2 · 2022-04-23T18:27:57Z

Thank you Gentlemen.....excellent

zas/install_intelr_graphic_linux_firmware.sh

UrbanVampire commented Jun 6, 2021 •

edited

Jiab77 commented Jun 9, 2021

Jiab77 commented Jun 9, 2021 •

edited

Jiab77 commented Jun 9, 2021 •

edited

UrbanVampire commented Jun 10, 2021

Jiab77 commented Jun 10, 2021 •

edited

UrbanVampire commented Jun 12, 2021 •

edited

UrbanVampire commented Jun 12, 2021

Jiab77 commented Jun 13, 2021

Jiab77 commented Jun 13, 2021

Jiab77 commented Jun 13, 2021

Jiab77 commented Jun 13, 2021 •

edited

Jiab77 commented Jun 13, 2021 •

edited

UrbanVampire commented Jun 14, 2021 •

edited

UrbanVampire commented Jun 14, 2021 •

edited

UrbanVampire commented Jun 14, 2021 •

edited

UrbanVampire commented Jun 15, 2021

UrbanVampire commented Jun 15, 2021

UrbanVampire commented Jun 15, 2021 •

edited

Jiab77 commented Jun 16, 2021

Jiab77 commented Jun 16, 2021 •

edited

Jiab77 commented Jun 17, 2021

Jiab77 commented Jun 17, 2021 •

edited

Jiab77 commented Jun 17, 2021 •

edited

UrbanVampire commented Jun 23, 2021

UrbanVampire commented Jun 23, 2021

UrbanVampire commented Jun 24, 2021 •

edited

Jiab77 commented Jun 28, 2021 •

edited

ErnestStCharly commented Jul 22, 2021

coitus2 commented Apr 23, 2022

	#!/bin/bash

	cd

	wget https://01.org/sites/default/files/downloads/intelr-graphics-linux/sklgucver61.tar.bz2 && \
	tar xvjf sklgucver61.tar.bz2 && cd skl_guc_ver6_1/ && sudo ./install.sh

	cd

	wget https://01.org/sites/default/files/downloads/intelr-graphics-linux/kbldmcver101.tar.bz2 && \
	tar xjvf kbldmcver101.tar.bz2 && cd kbl_dmc_ver1_01/ && sudo ./install.sh

	cd

	sudo update-initramfs -u -k all

zas/install_intelr_graphic_linux_firmware.sh

UrbanVampire commented Jun 6, 2021 • edited

Jiab77 commented Jun 9, 2021

Jiab77 commented Jun 9, 2021 • edited

Jiab77 commented Jun 9, 2021 • edited

UrbanVampire commented Jun 10, 2021

Jiab77 commented Jun 10, 2021 • edited

UrbanVampire commented Jun 12, 2021 • edited

UrbanVampire commented Jun 12, 2021

Jiab77 commented Jun 13, 2021

Jiab77 commented Jun 13, 2021

Jiab77 commented Jun 13, 2021

Jiab77 commented Jun 13, 2021 • edited

Jiab77 commented Jun 13, 2021 • edited

UrbanVampire commented Jun 14, 2021 • edited

UrbanVampire commented Jun 14, 2021 • edited

UrbanVampire commented Jun 14, 2021 • edited

UrbanVampire commented Jun 15, 2021

UrbanVampire commented Jun 15, 2021

UrbanVampire commented Jun 15, 2021 • edited

Jiab77 commented Jun 16, 2021

Jiab77 commented Jun 16, 2021 • edited

Jiab77 commented Jun 17, 2021

Jiab77 commented Jun 17, 2021 • edited

Jiab77 commented Jun 17, 2021 • edited

UrbanVampire commented Jun 23, 2021

UrbanVampire commented Jun 23, 2021

UrbanVampire commented Jun 24, 2021 • edited

Jiab77 commented Jun 28, 2021 • edited

ErnestStCharly commented Jul 22, 2021

coitus2 commented Apr 23, 2022

UrbanVampire commented Jun 6, 2021 •

edited

Jiab77 commented Jun 9, 2021 •

edited

Jiab77 commented Jun 9, 2021 •

edited

Jiab77 commented Jun 10, 2021 •

edited

UrbanVampire commented Jun 12, 2021 •

edited

Jiab77 commented Jun 13, 2021 •

edited

Jiab77 commented Jun 13, 2021 •

edited

UrbanVampire commented Jun 14, 2021 •

edited

UrbanVampire commented Jun 14, 2021 •

edited

UrbanVampire commented Jun 14, 2021 •

edited

UrbanVampire commented Jun 15, 2021 •

edited

Jiab77 commented Jun 16, 2021 •

edited

Jiab77 commented Jun 17, 2021 •

edited

Jiab77 commented Jun 17, 2021 •

edited

UrbanVampire commented Jun 24, 2021 •

edited

Jiab77 commented Jun 28, 2021 •

edited