Thursday, September 20, 2018

ZMQ error Installing Jupyter on nVidia Jetson XT2

When you try to install jupyter with
pip install jupyter
 on Jetson XT2, you run into an error saying zmq build failed (see below for logs...

This is because libzmq is not installled on Jetson by default.  To make it work, you just need to install libzmq :

sudo apt-get update
sudo apt-get install libzmq3-dev

That's it, do pip install jupyter afterwards, you will have jupyter running on your Jetson XT2 in no time. 



************************************************

Configure: Autodetecting ZMQ settings...
Custom ZMQ dir:
creating build/temp.linux-aarch64-3.5/scratch/tmp
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/timer_createo4b3hvp9.c -o build/temp.linux-aarch64-3.5/scratch/tmp/timer_createo4b3hvp9.o
/tmp/timer_createo4b3hvp9.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^
/tmp/timer_createo4b3hvp9.c: In function ‘main’:
/tmp/timer_createo4b3hvp9.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
timer_create();
^
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/scratch/tmp/timer_createo4b3hvp9.o -o build/temp.linux-aarch64-3.5/scratch/a.out
build/temp.linux-aarch64-3.5/scratch/tmp/timer_createo4b3hvp9.o: In function `main':
/tmp/timer_createo4b3hvp9.c:2: undefined reference to `timer_create'
collect2: error: ld returned 1 exit status
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-aarch64-3.5/scratch/vers.c -o build/temp.linux-aarch64-3.5/scratch/vers.o
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/scratch/vers.o -lzmq -lrt -o build/temp.linux-aarch64-3.5/scratch/vers
ZMQ version detected: 2.2.0

Bad libzmq version: Detected ZMQ version: 2.2.0, but require ZMQ >= 3.2
Explicitly allow legacy zmq by specifying `--zmq=/zmq/prefix`

Failed with default libzmq, trying again with /usr/local
{'libraries': ['zmq'], 'include_dirs': ['/usr/local/include'], 'runtime_library_dirs': ['/usr/local/lib'], 'library_dirs': ['/usr/local/lib'], 'extra_link_args': []}
************************************************
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir: /usr/local
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/timer_create6ro885h3.c -o build/temp.linux-aarch64-3.5/scratch/tmp/timer_create6ro885h3.o
/tmp/timer_create6ro885h3.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^
/tmp/timer_create6ro885h3.c: In function ‘main’:
/tmp/timer_create6ro885h3.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
timer_create();
^
Assembler messages:
Fatal error: can't create build/temp.linux-aarch64-3.5/scratch/tmp/timer_create6ro885h3.o: No such file or directory
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-aarch64-3.5/scratch/vers.c -o build/temp.linux-aarch64-3.5/scratch/vers.o
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/scratch/vers.o -L/usr/local/lib -Wl,--enable-new-dtags,-R/usr/local/lib -lzmq -lrt -o build/temp.linux-aarch64-3.5/scratch/vers
ZMQ version detected: 2.2.0

Bad libzmq version: Detected ZMQ version: 2.2.0, but require ZMQ >= 3.2
ZMQ_PREFIX=/usr/local
Explicitly allow legacy zmq by specifying `--zmq=/zmq/prefix`

************************************************
Warning: Couldn't find an acceptable libzmq on the system.

If you expected pyzmq to link against an installed libzmq, please check to make sure:

* You have a C compiler installed
* A development version of Python is installed (including headers)
* A development version of ZMQ >= 3.2 is installed (including headers)
* If ZMQ is not in a default location, supply the argument --zmq=
* If you did recently install ZMQ to a default location,
try rebuilding the ld cache with `sudo ldconfig`
or specify zmq's location with `--zmq=/usr/local`

You can skip all this detection/waiting nonsense if you know
you want pyzmq to bundle libzmq as an extension by passing:

`--zmq=bundled` ************************************************
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir:
creating build/temp.linux-aarch64-3.5/scratch/tmp
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/timer_createo4b3hvp9.c -o build/temp.linux-aarch64-3.5/scratch/tmp/timer_createo4b3hvp9.o
/tmp/timer_createo4b3hvp9.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^
/tmp/timer_createo4b3hvp9.c: In function ‘main’:
/tmp/timer_createo4b3hvp9.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
timer_create();
^
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/scratch/tmp/timer_createo4b3hvp9.o -o build/temp.linux-aarch64-3.5/scratch/a.out
build/temp.linux-aarch64-3.5/scratch/tmp/timer_createo4b3hvp9.o: In function `main':
/tmp/timer_createo4b3hvp9.c:2: undefined reference to `timer_create'
collect2: error: ld returned 1 exit status
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-aarch64-3.5/scratch/vers.c -o build/temp.linux-aarch64-3.5/scratch/vers.o
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/scratch/vers.o -lzmq -lrt -o build/temp.linux-aarch64-3.5/scratch/vers
ZMQ version detected: 2.2.0

Bad libzmq version: Detected ZMQ version: 2.2.0, but require ZMQ >= 3.2
Explicitly allow legacy zmq by specifying `--zmq=/zmq/prefix`

Failed with default libzmq, trying again with /usr/local
{'libraries': ['zmq'], 'include_dirs': ['/usr/local/include'], 'runtime_library_dirs': ['/usr/local/lib'], 'library_dirs': ['/usr/local/lib'], 'extra_link_args': []}
************************************************
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir: /usr/local
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/timer_create6ro885h3.c -o build/temp.linux-aarch64-3.5/scratch/tmp/timer_create6ro885h3.o
/tmp/timer_create6ro885h3.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^
/tmp/timer_create6ro885h3.c: In function ‘main’:
/tmp/timer_create6ro885h3.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
timer_create();
^
Assembler messages:
Fatal error: can't create build/temp.linux-aarch64-3.5/scratch/tmp/timer_create6ro885h3.o: No such file or directory
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-aarch64-3.5/scratch/vers.c -o build/temp.linux-aarch64-3.5/scratch/vers.o
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/scratch/vers.o -L/usr/local/lib -Wl,--enable-new-dtags,-R/usr/local/lib -lzmq -lrt -o build/temp.linux-aarch64-3.5/scratch/vers
ZMQ version detected: 2.2.0

Bad libzmq version: Detected ZMQ version: 2.2.0, but require ZMQ >= 3.2
ZMQ_PREFIX=/usr/local
Explicitly allow legacy zmq by specifying `--zmq=/zmq/prefix`

************************************************
Warning: Couldn't find an acceptable libzmq on the system.

If you expected pyzmq to link against an installed libzmq, please check to make sure:

* You have a C compiler installed
* A development version of Python is installed (including headers)
* A development version of ZMQ >= 3.2 is installed (including headers)
* If ZMQ is not in a default location, supply the argument --zmq=

* If you did recently install ZMQ to a default location,
try rebuilding the ld cache with `sudo ldconfig`
or specify zmq's location with `--zmq=/usr/local`

You can skip all this detection/waiting nonsense if you know
you want pyzmq to bundle libzmq as an extension by passing:

`--zmq=bundled`

I will now try to build libzmq as a Python extension
unless you interrupt me (^C) in the next 10 seconds...

1...
************************************************
Using bundled libzmq
already have bundled/zeromq
already have platform.hpp
************************************************
checking for timer_create
creating build/temp.linux-aarch64-3.5/tmp
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/timer_createvil73p56.c -o build/temp.linux-aarch64-3.5/tmp/timer_createvil73p56.o
/tmp/timer_createvil73p56.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^
/tmp/timer_createvil73p56.c: In function ‘main’:
/tmp/timer_createvil73p56.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
timer_create();
^
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/tmp/timer_createvil73p56.o -o build/temp.linux-aarch64-3.5/a.out
build/temp.linux-aarch64-3.5/tmp/timer_createvil73p56.o: In function `main':
/tmp/timer_createvil73p56.c:2: undefined reference to `timer_create'
collect2: error: ld returned 1 exit status
no timer_create, linking librt
************************************************
Fatal: Cython-generated file 'zmq/backend/cython/_device.c' not found.
Cython >= 0.20 is required to compile pyzmq from a development branch.
Please install Cython or download a release package of pyzmq.


----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-jx8cxtk1/pyzmq/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-9xas1kip-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-jx8cxtk1/pyzmq/
You are using pip version 8.1.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

I will now try to build libzmq as a Python extension
unless you interrupt me (^C) in the next 10 seconds...

1...
************************************************
Using bundled libzmq
already have bundled/zeromq
already have platform.hpp
************************************************
checking for timer_create
creating build/temp.linux-aarch64-3.5/tmp
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/timer_createvil73p56.c -o build/temp.linux-aarch64-3.5/tmp/timer_createvil73p56.o
/tmp/timer_createvil73p56.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^
/tmp/timer_createvil73p56.c: In function ‘main’:
/tmp/timer_createvil73p56.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
timer_create();
^
aarch64-linux-gnu-gcc -pthread build/temp.linux-aarch64-3.5/tmp/timer_createvil73p56.o -o build/temp.linux-aarch64-3.5/a.out
build/temp.linux-aarch64-3.5/tmp/timer_createvil73p56.o: In function `main':
/tmp/timer_createvil73p56.c:2: undefined reference to `timer_create'
collect2: error: ld returned 1 exit status
no timer_create, linking librt
************************************************
Fatal: Cython-generated file 'zmq/backend/cython/_device.c' not found.
Cython >= 0.20 is required to compile pyzmq from a development branch.
Please install Cython or download a release package of pyzmq.


----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-jx8cxtk1/pyzmq/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-9xas1kip-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-jx8cxtk1/pyzmq/
You are using pip version 8.1.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.



Tuesday, September 18, 2018

MacOS R Studio via AnaConda installing nloptr

If you installed AnaConda and used RStudio coming with it, You might have run into issues install.packages("car")
It would tell you Configuration errors trying to compile nloptr package telling you can't compile C++ code and ask you to try --host option.

Basically the failure is because car depends on nloptr package and newest nloptr package is only distribute as source (Mac binary is several years behind). Yet you can't compile C++ code from within R Studio (actually R) that was launched from Conda nevigator.

There is no solutions posted online although quite a few people ran into this issue.
The solution is to install nloptr using conda command (sudo)

Like:
conda install -c conda-forge r-nloptr 

You may need to sudo on MacOS (or Linux)

This command actually works on MacOS, Linux and windows.

Saturday, February 18, 2017

Get pip of python install to work behind proxy

OK, here is a simple task, use pip to install packages...
Other than if you are behind a proxy that uses https.

What happens is even though you issue the proper command line like below:
# pip install virtualenv --proxy http://user:pass@proxy:8080
you still get an error message like this:
.....
CertificateError: hostname 'proxy' doesn't match either of 'www.python.org', 'docs.python.org', 'bugs.python.org'.....

You tried to use HTTP_PROXY environment variable as well, still this Cert error??!!

Hmmmm.... what is going on?

It is because the ssl.py actually checks the hostname of the certificate subjectAltName, which would be python.org hostnames.  However, the 'hostname' it connected to is actually your proxy server, not the python servers.

OK, here is what you need to do - update the /usr/lib64/python2.7/ssl.py file
# vi /usr/lib64/pyton2.7/ssl.py

look for "if len(dnsnames) > 1:"

immediately before this line, you should insert below block:
value = 'proxy'
if _dnsname_match(value, hostname):
    return
dnsnames.append(value)

This is tested on python 2.7 behind a firewall.

Not all proxy servers will cause this problem, but if it does, here is the quick fix.