首页 > linux > azkaban 安装教程

azkaban 安装教程

2020年5月20日 mood 发表评论阅读评论

下载源码： git clone https://github.com/azkaban/azkaban.git
构建安装： cd azkaban; ./gradlew build installDist
启动服务： cd azkaban-solo-server/build/install/azkaban-solo-server; bin/start-solo.sh
停止服务：在azkaban-solo-server 安装到的目录执行 bin/azkaban-solo-shutdown.sh

第二步编译出错，没有装gradle，需要下载gradle，有没有网络或者下载卡死解决办法

[root@test azkaban]# ./gradlew build installDist
Downloading https://services.gradle.org/distributions/gradle-4.6-all.zip

Exception in thread "main" java.net.UnknownHostException: services.gradle.org
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
        at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.protocol.https.HttpsClient.<init>;(HttpsClient.java:264)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
        at org.gradle.wrapper.Download.downloadInternal(Download.java:66)
        at org.gradle.wrapper.Download.download(Download.java:51)
        at org.gradle.wrapper.Install$1.call(Install.java:62)
        at org.gradle.wrapper.Install$1.call(Install.java:48)
        at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:69)
        at org.gradle.wrapper.Install.createDist(Install.java:48)
        at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
        at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)

[root@test azkaban]# ./gradlew build installDist

Downloading https://services.gradle.org/distributions/gradle-4.6-all.zip

Exception in thread "main" java.net.UnknownHostException: services.gradle.org

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:589)

at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)

at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)

at sun.net.NetworkClient.doConnect(NetworkClient.java:180)

at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)

at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)

at sun.net.www.protocol.https.HttpsClient.<init>;(HttpsClient.java:264)

at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)

at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)

at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)

at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)

at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)

at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)

at org.gradle.wrapper.Download.downloadInternal(Download.java:66)

at org.gradle.wrapper.Download.download(Download.java:51)

at org.gradle.wrapper.Install$1.call(Install.java:62)

at org.gradle.wrapper.Install$1.call(Install.java:48)

at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:69)

at org.gradle.wrapper.Install.createDist(Install.java:48)

at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)

at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)

手动下载gradle，然后放到gradle/wrapper/

修改改配置文件gradle/wrapper/gradle-wrapper.properties

把distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip

改为
distributionUrl=gradle-4.6-all.zip

执行编译命令，出现如下错误，原因还是没有网络，gradle需要去远程仓库拉取文件。

[root@test azkaban]# ./gradlew build installDist
Downloading file:/root/azkaban/gradle/wrapper/gradle-4.6-all.zip
………………………………………………………………………………………..
Unzipping /root/.gradle/wrapper/dists/gradle-4.6-all/8i6ntuvrci3yhck4ezvhtqj4x/gradle-4.6-all.zip to /root/.gradle/wrapper/dists/gradle-4.6-all/8i6ntuvrci3yhck4ezvhtqj4x
Set executable permissions for: /root/.gradle/wrapper/dists/gradle-4.6-all/8i6ntuvrci3yhck4ezvhtqj4x/gradle-4.6/bin/gradle
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution with configuration on demand is an incubating feature.
FAILURE: Build failed with an exception.
Where:
Build file '/root/azkaban/build.gradle' line: 31
What went wrong:
Plugin [id: 'com.gradle.build-scan', version: '1.9'] was not found in any of the following sources:
Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
Plugin Repositories (could not resolve plugin artifact 'com.gradle.build-scan:com.gradle.build-scan.gradle.plugin:1.9')
Searched in the following repositories:
Gradle Central Plugin Repository
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org
BUILD FAILED in 6s

[root@test azkaban]# ./gradlew build installDist

Downloading file:/root/azkaban/gradle/wrapper/gradle-4.6-all.zip

………………………………………………………………………………………..

Unzipping /root/.gradle/wrapper/dists/gradle-4.6-all/8i6ntuvrci3yhck4ezvhtqj4x/gradle-4.6-all.zip to /root/.gradle/wrapper/dists/gradle-4.6-all/8i6ntuvrci3yhck4ezvhtqj4x

Set executable permissions for: /root/.gradle/wrapper/dists/gradle-4.6-all/8i6ntuvrci3yhck4ezvhtqj4x/gradle-4.6/bin/gradle

Starting a Gradle Daemon (subsequent builds will be faster)

Parallel execution with configuration on demand is an incubating feature.

FAILURE: Build failed with an exception.

Where:

Build file '/root/azkaban/build.gradle' line: 31

What went wrong:

Plugin [id: 'com.gradle.build-scan', version: '1.9'] was not found in any of the following sources:

Gradle Core Plugins (plugin is not in 'org.gradle' namespace)

Plugin Repositories (could not resolve plugin artifact 'com.gradle.build-scan:com.gradle.build-scan.gradle.plugin:1.9')

Searched in the following repositories:

Gradle Central Plugin Repository

Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 6s

gradle使用代理，
修改 gradle.propertiesj，加入如下内容。

systemProp.http.proxyHost=10.1.1.2
systemProp.http.proxyPort=8080
systemProp.https.proxyHost=10.1.1.2
systemProp.https.proxyPort=8080

systemProp.http.proxyHost=10.1.1.2

systemProp.http.proxyPort=8080

systemProp.https.proxyHost=10.1.1.2

systemProp.https.proxyPort=8080

使用国内镜像加快速度，把build.gradle的allprojects 替换为如下内容

allprojects {
    repositories {
        maven { url 'https://maven.aliyun.com/repository/public/' }
        mavenLocal()
        mavenCentral()
    }
}

allprojects {

repositories {

maven { url 'https://maven.aliyun.com/repository/public/' }

mavenLocal()

mavenCentral()

}

重新编译报错

[root@test azkaban]# ./gradlew build installDist
Parallel execution with configuration on demand is an incubating feature.

FAILURE: Build failed with an exception.

* Where:
Build file '/root/azkaban/build.gradle' line: 262

* What went wrong:
A problem occurred evaluating root project 'azkaban'.
> Could not get unknown property 'jacocoTestReport' for project ':az-core' of type org.gradle.api.Project.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 0s

[root@test azkaban]# ./gradlew build installDist

Parallel execution with configuration on demand is an incubating feature.

FAILURE: Build failed with an exception.

* Where:

Build file '/root/azkaban/build.gradle' line: 262

* What went wrong:

A problem occurred evaluating root project 'azkaban'.

> Could not get unknown property 'jacocoTestReport' for project ':az-core' of type org.gradle.api.Project.

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 0s

换aliyun镜像时参数删除了，apply plugin: 'jacoco'，需要加入

allprojects {
    apply plugin: 'jacoco'
    repositories {
        maven { url 'https://maven.aliyun.com/repository/public/' }
        mavenLocal()
        mavenCentral()
    }
}

allprojects {

apply plugin: 'jacoco'

repositories {

maven { url 'https://maven.aliyun.com/repository/public/' }

mavenLocal()

mavenCentral()

}

再次编译报错

[root@test azkaban]# ./gradlew build -x test
Parallel execution with configuration on demand is an incubating feature.

> Task :azkaban-web-server:npmSetup FAILED
npm ERR! code EPROTO
npm ERR! errno EPROTO
npm ERR! request to https://registry.npmjs.org/npm failed, reason: write EPROTO 139822221948800:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:
npm ERR!

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2019-07-16T05_16_11_768Z-debug.log


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':azkaban-web-server:npmSetup'.
> Process 'command '/root/azkaban/azkaban-web-server/build/nodejs/node-v8.10.0-linux-x64/bin/node'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s
67 actionable tasks: 6 executed, 61 up-to-date

[root@test azkaban]# ./gradlew build -x test

Parallel execution with configuration on demand is an incubating feature.

> Task :azkaban-web-server:npmSetup FAILED

npm ERR! code EPROTO

npm ERR! errno EPROTO

npm ERR! request to https://registry.npmjs.org/npm failed, reason: write EPROTO 139822221948800:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

npm ERR!

npm ERR! A complete log of this run can be found in:

npm ERR! /root/.npm/_logs/2019-07-16T05_16_11_768Z-debug.log

FAILURE: Build failed with an exception.

* What went wrong:

Execution failed for task ':azkaban-web-server:npmSetup'.

> Process 'command '/root/azkaban/azkaban-web-server/build/nodejs/node-v8.10.0-linux-x64/bin/node'' finished with non-zero exit value 1

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s

67 actionable tasks: 6 executed, 61 up-to-date

看了下--stacktrace，发下是openjdk中没有javafx，需要换成oracle的jdk

卸载java

 rpm -qa|grep jdk

1	rpm -qa\|grep jdk

yum -y remove java 上面命令显示的项

下载oraclejdk
https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

安装

rpm -ivh jdk-8u144-linux-x64.rpm

1	rpm -ivh jdk-8u144-linux-x64.rpm

手动配置环境变量：

编辑/etc/profile文件，在文件末尾加入以下内容
vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_144
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

export JAVA_HOME=/usr/java/jdk1.8.0_144

export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export PATH=$PATH:$JAVA_HOME/bin

使用source /etc/profile命令重新加载/etc/profile文件，使修改的内容在当前shell窗口中生效。（永久生效，重启系统或注销用户）

source /etc/profile

1	source /etc/profile

验证安装是否成功

java -version

或者通过命令设置

alternatives --config java

1	alternatives --config java

继续编译：报错

发现nodejs访问不了网络，设置nodejs使用代理

export NPM_CONFIG_HTTPS_PROXY=http://10.1.1.2:8080

总算编译成功

BUILD SUCCESSFUL in 41s
105 actionable tasks: 19 executed, 86 up-to-date

修改默认时区

tzselect #选择Asia/Shanghai

vim ./conf/azkaban.properties

default.timezone.id=Asia/Shanghai #修改时区

启动start-solo.sh

访问 http://ip:8081/ ，如果打不开需要开放防火墙端口

firewall-cmd --add-port 8081/tcp

https://azkaban.github.io/azkaban/docs/latest/#solo-setup

分类: linux 标签: azkaban

这几篇文章你可能也喜欢：

评论 (1) Trackbacks (0) 发表评论 Trackback

海王星三文鱼

2020年8月26日10:36 | #1

回复 | 引用

其实在缺少javafx包的部分，没有必要修改openjdk为oracle jdk的，openjdk仅仅是将jfx作为一个单独的包了，可以安装openjfx这个包就可以解决问题了。

本文目前尚无任何 trackbacks 和 pingbacks.

oracle 批量表授权 nignx 中的 ssl_ciphers 如何配置