MapReduce 应用程序主控 REST API

概述

MapReduce 应用程序主控 REST API 允许用户获取正在运行的 MapReduce 应用程序主控的状态。目前，这等同于正在运行的 MapReduce 作业。信息包括应用程序主控正在运行的作业以及所有作业详细信息，例如任务、计数器、配置、尝试等。应通过代理访问应用程序主控。此代理可配置为在资源管理器或单独的主机上运行。代理 URL 通常如下所示：http://proxy-http-address:port/proxy/appid。

MapReduce 应用程序主控信息 API

MapReduce 应用程序主控信息资源提供有关该 MapReduce 应用程序主控的总体信息。这包括应用程序 ID、启动时间、用户、名称等。

URI

以下两个 URI 都可为您提供 MapReduce 应用程序主控信息，其中应用程序 ID 由 appid 值标识。

支持的 HTTP 操作

  • GET

支持的查询参数

  None

info 对象的元素

当您请求 MapReduce 应用程序主控信息时，信息将作为 info 对象返回。

项目 数据类型 说明
appId 字符串 应用程序 ID
startedOn 长整型 应用程序启动的时间（自纪元以来的毫秒数）
name 字符串 应用程序名称
user 字符串 启动应用程序的用户名
elapsedTime 长整型 应用程序启动以来的时间（毫秒数）

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0003/ws/v1/mapreduce/info

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
  "info" : {
      "appId" : "application_1326232085508_0003",
      "startedOn" : 1326238244047,
      "user" : "user1",
      "name" : "Sleep job",
      "elapsedTime" : 32374
   }
}

XML 响应

HTTP 请求

  Accept: application/xml
  GET http://proxy-http-address:port/proxy/application_1326232085508_0003/ws/v1/mapreduce/info

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 223
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<info>
  <appId>application_1326232085508_0003</appId>
  <name>Sleep job</name>
  <user>user1</user>
  <startedOn>1326238244047</startedOn>
  <elapsedTime>32407</elapsedTime>
</info>

作业 API

作业资源提供在该应用程序主程序上运行的作业列表。另请参阅 作业 API 了解作业对象的语法。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  None

作业 对象的元素

当您请求作业列表时，信息将作为作业对象集合返回。另请参阅 作业 API 了解作业对象的语法。

项目 数据类型 说明
job 作业对象数组（JSON）/零个或多个作业对象（XML） 作业对象集合

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
  "jobs" : {
      "job" : [
         {
            "runningReduceAttempts" : 1,
            "reduceProgress" : 100,
            "failedReduceAttempts" : 0,
            "newMapAttempts" : 0,
            "mapsRunning" : 0,
            "state" : "RUNNING",
            "successfulReduceAttempts" : 0,
            "reducesRunning" : 1,
            "acls" : [
               {
                  "value" : " ",
                  "name" : "mapreduce.job.acl-modify-job"
               },
               {
                  "value" : " ",
                  "name" : "mapreduce.job.acl-view-job"
               }
            ],
            "reducesPending" : 0,
            "user" : "user1",
            "reducesTotal" : 1,
            "mapsCompleted" : 1,
            "startTime" : 1326238769379,
            "id" : "job_1326232085508_4_4",
            "successfulMapAttempts" : 1,
            "runningMapAttempts" : 0,
            "newReduceAttempts" : 0,
            "name" : "Sleep job",
            "mapsPending" : 0,
            "elapsedTime" : 59377,
            "reducesCompleted" : 0,
            "mapProgress" : 100,
            "diagnostics" : "",
            "failedMapAttempts" : 0,
            "killedReduceAttempts" : 0,
            "mapsTotal" : 1,
            "uberized" : false,
            "killedMapAttempts" : 0,
            "finishTime" : 0
         }
     ]
   }
 }

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 1214
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobs>
  <job>
    <startTime>1326238769379</startTime>
    <finishTime>0</finishTime>
    <elapsedTime>59416</elapsedTime>
    <id>job_1326232085508_4_4</id>
    <name>Sleep job</name>
    <user>user1</user>
    <state>RUNNING</state>
    <mapsTotal>1</mapsTotal>
    <mapsCompleted>1</mapsCompleted>
    <reducesTotal>1</reducesTotal>
    <reducesCompleted>0</reducesCompleted>
    <mapProgress>100.0</mapProgress>
    <reduceProgress>100.0</reduceProgress>
    <mapsPending>0</mapsPending>
    <mapsRunning>0</mapsRunning>
    <reducesPending>0</reducesPending>
    <reducesRunning>1</reducesRunning>
    <uberized>false</uberized>
    <diagnostics/>
    <newReduceAttempts>0</newReduceAttempts>
    <runningReduceAttempts>1</runningReduceAttempts>
    <failedReduceAttempts>0</failedReduceAttempts>
    <killedReduceAttempts>0</killedReduceAttempts>
    <successfulReduceAttempts>0</successfulReduceAttempts>
    <newMapAttempts>0</newMapAttempts>
    <runningMapAttempts>0</runningMapAttempts>
    <failedMapAttempts>0</failedMapAttempts>
    <killedMapAttempts>0</killedMapAttempts>
    <successfulMapAttempts>1</successfulMapAttempts>
    <acls>
      <name>mapreduce.job.acl-modify-job</name>
      <value> </value>
    </acls>
    <acls>
      <name>mapreduce.job.acl-view-job</name>
      <value> </value>
    </acls>
  </job>
</jobs>

作业 API

作业资源包含由该应用程序主程序启动的特定作业的信息。某些字段仅当用户拥有权限时才可访问 - 取决于 ACL 设置。

URI

使用以下 URI 获取作业对象，该对象由 jobid 值标识。

支持的 HTTP 操作

  • GET

支持的查询参数

  None

作业 对象的元素

项目 数据类型 说明
id 字符串 作业 ID
name 字符串 作业名称
user 字符串 用户名
state 字符串 作业状态 - 有效值为：NEW、INITED、RUNNING、SUCCEEDED、FAILED、KILL_WAIT、KILLED、ERROR
startTime 长整型 作业启动的时间（自纪元以来的毫秒数）
finishTime 长整型 作业完成的时间（自纪元以来的毫秒数）
elapsedTime 长整型 作业启动以来的经过时间（毫秒数）
mapsTotal 整数 映射总数
mapsCompleted 整数 已完成的映射数
reducesTotal 整数 还原总数
reducesCompleted 整数 已完成的还原数
diagnostics 字符串 诊断消息
uberized 布尔值 指示作业是否为 uber 作业 - 完全在应用程序主程序中运行
mapsPending 整数 仍需运行的映射数
mapsRunning 整数 正在运行的映射数
reducesPending 整数 仍需运行的还原数
reducesRunning 整数 正在运行的还原数
newReduceAttempts 整数 新的还原尝试次数
runningReduceAttempts 整数 正在运行的还原尝试次数
failedReduceAttempts 整数 失败的还原尝试次数
killedReduceAttempts 整数 已终止的还原尝试次数
successfulReduceAttempts 整数 成功减少尝试次数
newMapAttempts 整数 新映射尝试次数
runningMapAttempts 整数 正在运行的映射尝试次数
failedMapAttempts 整数 失败的映射尝试次数
killedMapAttempts 整数 已终止的映射尝试次数
successfulMapAttempts 整数 成功的映射尝试次数
acls acl 数组（json）/零个或多个 acl 对象（xml） acl 对象集合

acls 对象元素

项目 数据类型 说明
value 字符串 acl 值
name 字符串 acl 名称

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Server: Jetty(6.1.26)
  Content-Length: 720

响应正文

{
   "job" : {
      "runningReduceAttempts" : 1,
      "reduceProgress" : 100,
      "failedReduceAttempts" : 0,
      "newMapAttempts" : 0,
      "mapsRunning" : 0,
      "state" : "RUNNING",
      "successfulReduceAttempts" : 0,
      "reducesRunning" : 1,
      "acls" : [
         {
            "value" : " ",
            "name" : "mapreduce.job.acl-modify-job"
         },
         {
            "value" : " ",
            "name" : "mapreduce.job.acl-view-job"
         }
      ],
      "reducesPending" : 0,
      "user" : "user1",
      "reducesTotal" : 1,
      "mapsCompleted" : 1,
      "startTime" : 1326238769379,
      "id" : "job_1326232085508_4_4",
      "successfulMapAttempts" : 1,
      "runningMapAttempts" : 0,
      "newReduceAttempts" : 0,
      "name" : "Sleep job",
      "mapsPending" : 0,
      "elapsedTime" : 59437,
      "reducesCompleted" : 0,
      "mapProgress" : 100,
      "diagnostics" : "",
      "failedMapAttempts" : 0,
      "killedReduceAttempts" : 0,
      "mapsTotal" : 1,
      "uberized" : false,
      "killedMapAttempts" : 0,
      "finishTime" : 0
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 1201
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<job>
  <startTime>1326238769379</startTime>
  <finishTime>0</finishTime>
  <elapsedTime>59474</elapsedTime>
  <id>job_1326232085508_4_4</id>
  <name>Sleep job</name>
  <user>user1</user>
  <state>RUNNING</state>
  <mapsTotal>1</mapsTotal>
  <mapsCompleted>1</mapsCompleted>
  <reducesTotal>1</reducesTotal>
  <reducesCompleted>0</reducesCompleted>
  <mapProgress>100.0</mapProgress>
  <reduceProgress>100.0</reduceProgress>
  <mapsPending>0</mapsPending>
  <mapsRunning>0</mapsRunning>
  <reducesPending>0</reducesPending>
  <reducesRunning>1</reducesRunning>
  <uberized>false</uberized>
  <diagnostics/>
  <newReduceAttempts>0</newReduceAttempts>
  <runningReduceAttempts>1</runningReduceAttempts>
  <failedReduceAttempts>0</failedReduceAttempts>
  <killedReduceAttempts>0</killedReduceAttempts>
  <successfulReduceAttempts>0</successfulReduceAttempts>
  <newMapAttempts>0</newMapAttempts>
  <runningMapAttempts>0</runningMapAttempts>
  <failedMapAttempts>0</failedMapAttempts>
  <killedMapAttempts>0</killedMapAttempts>
  <successfulMapAttempts>1</successfulMapAttempts>
  <acls>
    <name>mapreduce.job.acl-modify-job</name>
    <value> </value>
  </acls>
  <acls>
    <name>mapreduce.job.acl-view-job</name>    <value> </value>
  </acls>
</job>

作业尝试 API

使用作业尝试 API，您可以获取表示作业尝试的资源集合。当您对该资源运行 GET 操作时，您将获取作业尝试对象集合。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  None

jobAttempts 对象元素

当您请求作业尝试列表时，信息将作为作业尝试对象数组返回。

项目 数据类型 说明
jobAttempt 作业尝试对象数组（JSON）/零个或多个作业尝试对象（XML） 作业尝试对象集合

jobAttempt 对象元素

项目 数据类型 说明
id 字符串 作业尝试 ID
nodeId 字符串 尝试运行所在的节点的节点 ID
nodeHttpAddress 字符串 尝试运行所在的节点的节点 http 地址
logsLink 字符串 作业尝试日志的 http 链接
containerId 字符串 作业尝试的容器 ID
startTime 长整型 尝试的开始时间（自纪元以来的毫秒数）

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/jobattempts

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "jobAttempts" : {
      "jobAttempt" : [
         {
            "nodeId" : "host.domain.com:8041",
            "nodeHttpAddress" : "host.domain.com:8042",
            "startTime" : 1326238773493,
            "id" : 1,
            "logsLink" : "http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001",
            "containerId" : "container_1326232085508_0004_01_000001"
         }
      ]
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/jobattempts
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 498
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobAttempts>
  <jobAttempt>
    <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress>
    <nodeId>host.domain.com:8041</nodeId>
    <id>1</id>
    <startTime>1326238773493</startTime>
    <containerId>container_1326232085508_0004_01_000001</containerId>
    <logsLink>http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001</logsLink>
  </jobAttempt>
</jobAttempts>

作业计数器 API

使用作业计数器 API，您可以获取表示该作业所有计数器的资源集合。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  None

jobCounters 对象元素

项目 数据类型 说明
id 字符串 作业 ID
counterGroup counterGroup 对象数组（JSON）/零个或多个 counterGroup 对象（XML） counter 组对象集合

counterGroup 对象元素

项目 数据类型 说明
counterGroupName 字符串 counter 组名称
counter counter 对象数组（JSON）/零个或多个 counter 对象（XML） counter 对象集合

counter 对象元素

项目 数据类型 说明
name 字符串 counter 名称
reduceCounterValue 长整型 reduce 任务的 counter 值
mapCounterValue 长整型 map 任务的 counter 值
totalCounterValue 长整型 所有任务的 counter 值

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/counters

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "jobCounters" : {
      "id" : "job_1326232085508_4_4",
      "counterGroup" : [
         {
            "counterGroupName" : "Shuffle Errors",
            "counter" : [
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "BAD_ID"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "CONNECTION"
               },
              {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "IO_ERROR"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "WRONG_LENGTH"
               },                {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "WRONG_MAP"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "WRONG_REDUCE"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.FileSystemCounter",
            "counter" : [
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 2483,
                  "name" : "FILE_BYTES_READ"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 108763,
                  "name" : "FILE_BYTES_WRITTEN"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "FILE_READ_OPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "FILE_LARGE_READ_OPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "FILE_WRITE_OPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 48,
                  "name" : "HDFS_BYTES_READ"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "HDFS_BYTES_WRITTEN"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1,
                  "name" : "HDFS_READ_OPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "HDFS_LARGE_READ_OPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "HDFS_WRITE_OPS"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.TaskCounter",
            "counter" : [
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1,
                  "name" : "MAP_INPUT_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1200,
                  "name" : "MAP_OUTPUT_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 4800,
                  "name" : "MAP_OUTPUT_BYTES"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 2235,
                  "name" : "MAP_OUTPUT_MATERIALIZED_BYTES"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 48,
                  "name" : "SPLIT_RAW_BYTES"
               },
              {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "COMBINE_INPUT_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "COMBINE_OUTPUT_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 460,
                  "name" : "REDUCE_INPUT_GROUPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 2235,
                  "name" : "REDUCE_SHUFFLE_BYTES"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 460,
                  "name" : "REDUCE_INPUT_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "REDUCE_OUTPUT_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1200,
                  "name" : "SPILLED_RECORDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1,
                  "name" : "SHUFFLED_MAPS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "FAILED_SHUFFLE"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1,
                  "name" : "MERGED_MAP_OUTPUTS"
               },                {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 58,
                  "name" : "GC_TIME_MILLIS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 1580,
                  "name" : "CPU_MILLISECONDS"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 462643200,
                  "name" : "PHYSICAL_MEMORY_BYTES"
               },
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 2149728256,
                  "name" : "VIRTUAL_MEMORY_BYTES"
               },
              {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 357957632,
                  "name" : "COMMITTED_HEAP_BYTES"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter",
            "counter" : [
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "BYTES_READ"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter",
            "counter" : [
               {
                  "reduceCounterValue" : 0,
                  "mapCounterValue" : 0,
                  "totalCounterValue" : 0,
                  "name" : "BYTES_WRITTEN"
               }
            ]
         }
      ]
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/counters
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 7027
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobCounters>
  <id>job_1326232085508_4_4</id>
  <counterGroup>
    <counterGroupName>Shuffle Errors</counterGroupName>
    <counter>
      <name>BAD_ID</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>CONNECTION</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>IO_ERROR</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>WRONG_LENGTH</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>WRONG_MAP</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>WRONG_REDUCE</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
  </counterGroup>
  <counterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.FileSystemCounter</counterGroupName>
    <counter>
      <name>FILE_BYTES_READ</name>
      <totalCounterValue>2483</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>FILE_BYTES_WRITTEN</name>
      <totalCounterValue>108763</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>FILE_READ_OPS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>FILE_LARGE_READ_OPS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>FILE_WRITE_OPS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>HDFS_BYTES_READ</name>
      <totalCounterValue>48</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>HDFS_BYTES_WRITTEN</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>HDFS_READ_OPS</name>
      <totalCounterValue>1</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>HDFS_LARGE_READ_OPS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>HDFS_WRITE_OPS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
  </counterGroup>
  <counterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.TaskCounter</counterGroupName>
    <counter>
      <name>MAP_INPUT_RECORDS</name>
      <totalCounterValue>1</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>MAP_OUTPUT_RECORDS</name>
      <totalCounterValue>1200</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>MAP_OUTPUT_BYTES</name>
      <totalCounterValue>4800</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>MAP_OUTPUT_MATERIALIZED_BYTES</name>
      <totalCounterValue>2235</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>SPLIT_RAW_BYTES</name>
      <totalCounterValue>48</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>COMBINE_INPUT_RECORDS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>COMBINE_OUTPUT_RECORDS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>REDUCE_INPUT_GROUPS</name>
      <totalCounterValue>460</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>REDUCE_SHUFFLE_BYTES</name>
      <totalCounterValue>2235</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>REDUCE_INPUT_RECORDS</name>
      <totalCounterValue>460</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>REDUCE_OUTPUT_RECORDS</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>SPILLED_RECORDS</name>
      <totalCounterValue>1200</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>SHUFFLED_MAPS</name>
      <totalCounterValue>1</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>FAILED_SHUFFLE</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>MERGED_MAP_OUTPUTS</name>
      <totalCounterValue>1</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>GC_TIME_MILLIS</name>
      <totalCounterValue>58</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>CPU_MILLISECONDS</name>
      <totalCounterValue>1580</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>PHYSICAL_MEMORY_BYTES</name>
      <totalCounterValue>462643200</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>VIRTUAL_MEMORY_BYTES</name>
      <totalCounterValue>2149728256</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
    <counter>
      <name>COMMITTED_HEAP_BYTES</name>
      <totalCounterValue>357957632</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
  </counterGroup>
  <counterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter</counterGroupName>
    <counter>
      <name>BYTES_READ</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>  </counterGroup>  <counterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter</counterGroupName>
    <counter>      <name>BYTES_WRITTEN</name>
      <totalCounterValue>0</totalCounterValue>
      <mapCounterValue>0</mapCounterValue>
      <reduceCounterValue>0</reduceCounterValue>
    </counter>
  </counterGroup>
</jobCounters>

作业配置 API

作业配置资源包含有关此作业的作业配置信息。

URI

使用以下 URI 从由 jobid 值标识的作业获取作业配置信息。

支持的 HTTP 操作

  • GET

支持的查询参数

  None

conf 对象的元素

项目 数据类型 说明
path 字符串 作业配置文件的路径
property 配置属性数组 (JSON)/零个或多个属性对象 (XML) 属性对象集合

property 对象的元素

项目 数据类型 说明
name 字符串 配置属性的名称
value 字符串 配置属性的值
source 字符串 此配置对象来自的位置。如果有多个，则会显示历史记录，列表末尾是最新来源。

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/conf

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

这是输出的一个小片段，因为输出非常大。实际输出包含作业配置文件中的每个属性。

{
   "conf" : {
      "path" : "hdfs://host.domain.com:9000/user/user1/.staging/job_1326232085508_0004/job.xml",
      "property" : [
         {
            "value" : "/home/hadoop/hdfs/data",
            "name" : "dfs.datanode.data.dir",
            "source" : ["hdfs-site.xml", "job.xml"]
         },
         {
            "value" : "org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer",
            "name" : "hadoop.http.filter.initializers"
            "source" : ["programmatically", "job.xml"]
         },
         {
            "value" : "/home/hadoop/tmp",
            "name" : "mapreduce.cluster.temp.dir"
            "source" : ["mapred-site.xml"]
         },
         ...
      ]
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/conf
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 552
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<conf>
  <path>hdfs://host.domain.com:9000/user/user1/.staging/job_1326232085508_0004/job.xml</path>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>/home/hadoop/hdfs/data</value>
    <source>hdfs-site.xml</source>
    <source>job.xml</source>
  </property>
  <property>
    <name>hadoop.http.filter.initializers</name>
    <value>org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer</value>
    <source>programmatically</source>
    <source>job.xml</source>
  </property>
  <property>
    <name>mapreduce.cluster.temp.dir</name>
    <value>/home/hadoop/tmp</value>
    <source>mapred-site.xml</source>
  </property>
  ...
</conf>

任务 API

使用任务 API，您可以获取表示作业所有任务的资源集合。在此资源上运行 GET 操作时，您将获得任务对象集合。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  • type - 任务类型，有效值为 m 或 r。m 表示映射任务，r 表示归约任务。

tasks 对象的元素

当您请求任务列表时，信息将作为任务对象数组返回。另请参阅 任务 API 了解任务对象的语法。

项目 数据类型 说明
task 任务对象数组 (JSON)/零个或多个任务对象 (XML) 任务对象集合

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "tasks" : {
      "task" : [
         {
            "progress" : 100,
            "elapsedTime" : 2768,
            "state" : "SUCCEEDED",
            "startTime" : 1326238773493,
            "id" : "task_1326232085508_4_4_m_0",
            "type" : "MAP",
            "successfulAttempt" : "attempt_1326232085508_4_4_m_0_0",
            "finishTime" : 1326238776261
         },
         {
            "progress" : 100,
            "elapsedTime" : 0,
            "state" : "RUNNING",
            "startTime" : 1326238777460,
            "id" : "task_1326232085508_4_4_r_0",
            "type" : "REDUCE",
            "successfulAttempt" : "",
            "finishTime" : 0
         }
      ]
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 603
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tasks>
  <task>
    <startTime>1326238773493</startTime>
    <finishTime>1326238776261</finishTime>
    <elapsedTime>2768</elapsedTime>
    <progress>100.0</progress>
    <id>task_1326232085508_4_4_m_0</id>
    <state>SUCCEEDED</state>
    <type>MAP</type>
    <successfulAttempt>attempt_1326232085508_4_4_m_0_0</successfulAttempt>
  </task>
  <task>
    <startTime>1326238777460</startTime>
    <finishTime>0</finishTime>
    <elapsedTime>0</elapsedTime>
    <progress>100.0</progress>
    <id>task_1326232085508_4_4_r_0</id>
    <state>RUNNING</state>
    <type>REDUCE</type>
    <successfulAttempt/>
  </task>
</tasks>

任务 API

任务资源包含有关作业中特定任务的信息。

URI

使用以下 URI 从由 taskid 值标识的任务获取任务对象。

支持的 HTTP 操作

  • GET

支持的查询参数

  None

task 对象的元素

项目 数据类型 说明
id 字符串 任务 ID
state 字符串 任务状态 - 有效值为：NEW、SCHEDULED、RUNNING、SUCCEEDED、FAILED、KILL_WAIT、KILLED
type 字符串 任务类型 - MAP 或 REDUCE
successfulAttempt 字符串 上次成功尝试的 ID
progress float 任务进度（百分比）
startTime 长整型 任务开始时间（自纪元以来的毫秒数）
finishTime 长整型 任务结束时间（自纪元以来的毫秒数）
elapsedTime 长整型 自应用程序启动以来的经过时间（毫秒）

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "task" : {
      "progress" : 100,
      "elapsedTime" : 0,
      "state" : "RUNNING",
      "startTime" : 1326238777460,
      "id" : "task_1326232085508_4_4_r_0",
      "type" : "REDUCE",
      "successfulAttempt" : "",
      "finishTime" : 0
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 299
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<task>
  <startTime>1326238777460</startTime>
  <finishTime>0</finishTime>
  <elapsedTime>0</elapsedTime>
  <progress>100.0</progress>
  <id>task_1326232085508_4_4_r_0</id>
  <state>RUNNING</state>
  <type>REDUCE</type>
  <successfulAttempt/>
</task>

任务计数器 API

使用任务计数器 API，您可以将一组资源对象化，这些资源表示该任务的所有计数器。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  None

jobTaskCounters 对象的元素

项目 数据类型 说明
id 字符串 任务 ID
taskcounterGroup counterGroup 对象数组（JSON）/零个或多个 counterGroup 对象（XML） counter 组对象集合

counterGroup 对象元素

项目 数据类型 说明
counterGroupName 字符串 counter 组名称
counter counter 对象数组（JSON）/零个或多个 counter 对象（XML） counter 对象集合

counter 对象元素

项目 数据类型 说明
name 字符串 counter 名称
value 长整型 计数器的值

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/counters

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "jobTaskCounters" : {
      "id" : "task_1326232085508_4_4_r_0",
      "taskCounterGroup" : [
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.FileSystemCounter",
            "counter" : [
               {
                  "value" : 2363,
                  "name" : "FILE_BYTES_READ"
               },
               {
                  "value" : 54372,
                  "name" : "FILE_BYTES_WRITTEN"
               },
               {
                  "value" : 0,
                  "name" : "FILE_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "FILE_LARGE_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "FILE_WRITE_OPS"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_BYTES_READ"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_BYTES_WRITTEN"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_LARGE_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_WRITE_OPS"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.TaskCounter",
            "counter" : [
               {
                  "value" : 0,
                  "name" : "COMBINE_INPUT_RECORDS"
               },
               {
                  "value" : 0,
                  "name" : "COMBINE_OUTPUT_RECORDS"
               },
               {
                  "value" : 460,
                  "name" : "REDUCE_INPUT_GROUPS"
               },
               {
                  "value" : 2235,
                  "name" : "REDUCE_SHUFFLE_BYTES"
               },
               {
                  "value" : 460,
                  "name" : "REDUCE_INPUT_RECORDS"
               },
               {
                  "value" : 0,
                  "name" : "REDUCE_OUTPUT_RECORDS"
               },
               {
                  "value" : 0,
                  "name" : "SPILLED_RECORDS"
               },
               {
                  "value" : 1,
                  "name" : "SHUFFLED_MAPS"
               },
               {
                  "value" : 0,
                  "name" : "FAILED_SHUFFLE"
               },
               {
                  "value" : 1,
                  "name" : "MERGED_MAP_OUTPUTS"
               },
               {
                  "value" : 26,
                  "name" : "GC_TIME_MILLIS"
               },
               {
                  "value" : 860,
                  "name" : "CPU_MILLISECONDS"
               },
               {
                  "value" : 107839488,
                  "name" : "PHYSICAL_MEMORY_BYTES"
               },
               {
                  "value" : 1123147776,
                  "name" : "VIRTUAL_MEMORY_BYTES"
               },
               {
                  "value" : 57475072,
                  "name" : "COMMITTED_HEAP_BYTES"
               }
            ]
         },
         {
            "counterGroupName" : "Shuffle Errors",
            "counter" : [
               {
                  "value" : 0,
                  "name" : "BAD_ID"
               },
               {
                  "value" : 0,
                  "name" : "CONNECTION"
               },
               {
                  "value" : 0,
                  "name" : "IO_ERROR"
               },
               {
                  "value" : 0,
                  "name" : "WRONG_LENGTH"
               },
               {
                  "value" : 0,
                  "name" : "WRONG_MAP"
               },
               {
                  "value" : 0,
                  "name" : "WRONG_REDUCE"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter",
            "counter" : [
               {
                  "value" : 0,
                  "name" : "BYTES_WRITTEN"
               }
            ]
         }
      ]
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/counters
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 2660
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskCounters>
  <id>task_1326232085508_4_4_r_0</id>
  <taskCounterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.FileSystemCounter</counterGroupName>
    <counter>
      <name>FILE_BYTES_READ</name>
      <value>2363</value>
    </counter>
    <counter>
      <name>FILE_BYTES_WRITTEN</name>
      <value>54372</value>
    </counter>
    <counter>
      <name>FILE_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>FILE_LARGE_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>FILE_WRITE_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_BYTES_READ</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_BYTES_WRITTEN</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_LARGE_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_WRITE_OPS</name>
      <value>0</value>
    </counter>
  </taskCounterGroup>
  <taskCounterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.TaskCounter</counterGroupName>
    <counter>
      <name>COMBINE_INPUT_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>COMBINE_OUTPUT_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>REDUCE_INPUT_GROUPS</name>
      <value>460</value>
    </counter>
    <counter>
      <name>REDUCE_SHUFFLE_BYTES</name>
      <value>2235</value>
    </counter>
    <counter>
      <name>REDUCE_INPUT_RECORDS</name>
      <value>460</value>
    </counter>
    <counter>
      <name>REDUCE_OUTPUT_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>SPILLED_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>SHUFFLED_MAPS</name>
      <value>1</value>
    </counter>
    <counter>
      <name>FAILED_SHUFFLE</name>
      <value>0</value>
    </counter>
    <counter>
      <name>MERGED_MAP_OUTPUTS</name>
      <value>1</value>
    </counter>
    <counter>
      <name>GC_TIME_MILLIS</name>
      <value>26</value>
    </counter>
    <counter>
      <name>CPU_MILLISECONDS</name>
      <value>860</value>
    </counter>
    <counter>
      <name>PHYSICAL_MEMORY_BYTES</name>
      <value>107839488</value>
    </counter>
    <counter>
      <name>VIRTUAL_MEMORY_BYTES</name>
      <value>1123147776</value>
    </counter>
    <counter>
      <name>COMMITTED_HEAP_BYTES</name>
      <value>57475072</value>
    </counter>
  </taskCounterGroup>
  <taskCounterGroup>
    <counterGroupName>Shuffle Errors</counterGroupName>
    <counter>
      <name>BAD_ID</name>
      <value>0</value>
    </counter>
    <counter>
      <name>CONNECTION</name>
      <value>0</value>
    </counter>
    <counter>
      <name>IO_ERROR</name>
      <value>0</value>
    </counter>
    <counter>
      <name>WRONG_LENGTH</name>
      <value>0</value>
    </counter>
    <counter>
      <name>WRONG_MAP</name>
      <value>0</value>
    </counter>
    <counter>
      <name>WRONG_REDUCE</name>
      <value>0</value>
    </counter>
  </taskCounterGroup>
  <taskCounterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter</counterGroupName>
    <counter>
      <name>BYTES_WRITTEN</name>
      <value>0</value>
    </counter>
  </taskCounterGroup>
</jobTaskCounters>

任务尝试 API

使用任务尝试 API，您可以获取一组资源，这些资源表示作业中的任务尝试。当您对该资源运行 GET 操作时，您将获取一组任务尝试对象。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  None

taskAttempts 对象的元素

当您请求任务尝试列表时，信息将作为任务尝试对象数组返回。另请参阅 任务尝试 API 以了解任务对象的语法。

项目 数据类型 说明
taskAttempt 任务尝试对象数组 (JSON)/零个或多个任务尝试对象 (XML) 任务尝试对象集合

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "taskAttempts" : {
      "taskAttempt" : [
         {
            "elapsedMergeTime" : 47,
            "shuffleFinishTime" : 1326238780052,
            "assignedContainerId" : "container_1326232085508_0004_01_000003",
            "progress" : 100,
            "elapsedTime" : 0,
            "state" : "RUNNING",
            "elapsedShuffleTime" : 2592,
            "mergeFinishTime" : 1326238780099,
            "rack" : "/98.139.92.0",
            "elapsedReduceTime" : 0,
            "nodeHttpAddress" : "host.domain.com:8042",
            "type" : "REDUCE",
            "startTime" : 1326238777460,
            "id" : "attempt_1326232085508_4_4_r_0_0",
            "finishTime" : 0
         }
      ]
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 807
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<taskAttempts>
  <taskAttempt>
    <startTime>1326238777460</startTime>
    <finishTime>0</finishTime>
    <elapsedTime>0</elapsedTime>
    <progress>100.0</progress>
    <id>attempt_1326232085508_4_4_r_0_0</id>
    <rack>/98.139.92.0</rack>
    <state>RUNNING</state>
    <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress>
    <type>REDUCE</type>
    <assignedContainerId>container_1326232085508_0004_01_000003</assignedContainerId>
    <shuffleFinishTime>1326238780052</shuffleFinishTime>
    <mergeFinishTime>1326238780099</mergeFinishTime>
    <elapsedShuffleTime>2592</elapsedShuffleTime>
    <elapsedMergeTime>47</elapsedMergeTime>
    <elapsedReduceTime>0</elapsedReduceTime>
  </taskAttempt>
</taskAttempts>

任务尝试 API

任务尝试资源包含有关作业中特定任务尝试的信息。

URI

使用以下 URI 从由 attemptid 值标识的任务中获取任务尝试对象。

支持的 HTTP 操作

  • GET

支持的查询参数

  None

taskAttempt 对象的元素

项目 数据类型 说明
id 字符串 任务 ID
rack 字符串 机架
state 字符串 任务尝试的状态 - 有效值包括：NEW、UNASSIGNED、ASSIGNED、RUNNING、COMMIT_PENDING、SUCCESS_CONTAINER_CLEANUP、SUCCEEDED、FAIL_CONTAINER_CLEANUP、FAIL_TASK_CLEANUP、FAILED、KILL_CONTAINER_CLEANUP、KILL_TASK_CLEANUP、KILLED
type 字符串 任务类型
assignedContainerId 字符串 此尝试分配到的容器 ID
nodeHttpAddress 字符串 此任务尝试在上面运行的节点的 HTTP 地址
diagnostics 字符串 诊断消息
progress float 任务尝试进度（百分比）
startTime 长整型 任务尝试开始时间（自纪元以来的毫秒数）
finishTime 长整型 任务尝试结束时间（自纪元以来的毫秒数）
elapsedTime 长整型 自任务尝试开始以来的经过时间（毫秒）

对于 reduce 任务尝试，您还有以下字段

项目 数据类型 说明
shuffleFinishTime 长整型 shuffle 完成的时间（自纪元以来的毫秒数）
mergeFinishTime 长整型 合并完成的时间（自纪元以来的毫秒数）
elapsedShuffleTime 长整型 洗牌阶段完成所需时间（以毫秒为单位，从 reduce 任务开始到洗牌结束的时间）
elapsedMergeTime 长整型 合并阶段完成所需时间（以毫秒为单位，从洗牌结束到合并结束的时间）
elapsedReduceTime 长整型 reduce 阶段完成所需时间（以毫秒为单位，从合并结束到 reduce 任务结束的时间）

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "taskAttempt" : {
      "elapsedMergeTime" : 47,
      "shuffleFinishTime" : 1326238780052,
      "assignedContainerId" : "container_1326232085508_0004_01_000003",
      "progress" : 100,
      "elapsedTime" : 0,
      "state" : "RUNNING",
      "elapsedShuffleTime" : 2592,
      "mergeFinishTime" : 1326238780099,
      "rack" : "/98.139.92.0",
      "elapsedReduceTime" : 0,
      "nodeHttpAddress" : "host.domain.com:8042",
      "startTime" : 1326238777460,
      "id" : "attempt_1326232085508_4_4_r_0_0",
      "type" : "REDUCE",
      "finishTime" : 0
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 691
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<taskAttempt>
  <startTime>1326238777460</startTime>
  <finishTime>0</finishTime>
  <elapsedTime>0</elapsedTime>
  <progress>100.0</progress>
  <id>attempt_1326232085508_4_4_r_0_0</id>
  <rack>/98.139.92.0</rack>
  <state>RUNNING</state>
  <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress>
  <type>REDUCE</type>
  <assignedContainerId>container_1326232085508_0004_01_000003</assignedContainerId>
  <shuffleFinishTime>1326238780052</shuffleFinishTime>
  <mergeFinishTime>1326238780099</mergeFinishTime>
  <elapsedShuffleTime>2592</elapsedShuffleTime>
  <elapsedMergeTime>47</elapsedMergeTime>
  <elapsedReduceTime>0</elapsedReduceTime>
</taskAttempt>

任务尝试状态 API

使用任务尝试状态 API，您可以查询已提交任务尝试的状态，还可以通过使用 PUT 请求（状态设置为“KILLED”）修改正在运行的任务尝试的状态来终止正在运行的任务尝试。要执行 PUT 操作，必须为 AM Web 服务设置身份验证。此外，您必须有权终止任务尝试。目前，您只能将状态更改为“KILLED”；尝试将状态更改为任何其他状态都会导致 400 错误响应。下面是未经授权和错误请求错误的示例。执行成功的 PUT 后，初始响应可能是 202。您可以通过重复 PUT 请求（直到收到 200）、使用 GET 方法查询状态或查询任务尝试信息并检查状态来确认应用程序已终止。在以下示例中，我们重复 PUT 请求并收到 200 响应。

请注意，要终止任务尝试，您必须为 HTTP 接口设置身份验证过滤器。此功能要求在 HttpServletRequest 中设置用户名。如果未设置过滤器，则响应将为“UNAUTHORIZED”响应。

此功能目前处于 alpha 阶段，将来可能会发生更改。

URI

支持的 HTTP 操作

  • GET
    • POST

支持的查询参数

  None

jobTaskAttemptState 对象的元素

当您请求应用程序的状态时，返回的信息包含以下字段

项目 数据类型 说明
state 字符串 应用程序状态 - 可以是“NEW”、“STARTING”、“RUNNING”、“COMMIT_PENDING”、“SUCCEEDED”、“FAILED”、“KILLED”之一

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state

响应头

HTTP/1.1 200 OK
Content-Type: application/json
Server: Jetty(6.1.26)
Content-Length: 20

响应正文

{
  "state":"STARTING"
}

HTTP 请求

  PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state

请求正文

{
  "state":"KILLED"
}

响应头

HTTP/1.1 200 OK
Content-Type: application/json
Server: Jetty(6.1.26)
Content-Length: 18

响应正文

{
  "state":"KILLED"
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state

响应头

HTTP/1.1 200 OK
Content-Type: application/xml
Server: Jetty(6.1.26)
Content-Length: 121

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptState>
  <state>STARTING</state>
</jobTaskAttemptState>

HTTP 请求

  PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state

请求正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptState>
  <state>KILLED</state>
</jobTaskAttemptState>

响应头

HTTP/1.1 200 OK
Content-Type: application/xml
Server: Jetty(6.1.26)
Content-Length: 121

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptState>
  <state>KILLED</state>
</jobTaskAttemptState>

未经授权的错误响应

HTTP 请求

  PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state

请求正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptState>
  <state>KILLED</state>
</jobTaskAttemptState>

响应头

HTTP/1.1 403 Unauthorized
Content-Type: application/json
Server: Jetty(6.1.26)

错误请求错误响应

HTTP 请求

  PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state

请求正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptState>
  <state>RUNNING</state>
</jobTaskAttemptState>

响应头

HTTP/1.1 400
Content-Length: 295
Content-Type: application/xml
Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RemoteException>
  <exception>BadRequestException</exception>
  <message>java.lang.Exception: Only 'KILLED' is allowed as a target state.</message>
  <javaClassName>org.apache.hadoop.yarn.webapp.BadRequestException</javaClassName>
</RemoteException>

任务尝试计数器 API

使用任务尝试计数器 API，您可以对象一个资源集合，这些资源表示该任务尝试的所有计数器。

URI

支持的 HTTP 操作

  • GET

支持的查询参数

  None

jobTaskAttemptCounters 对象的元素

项目 数据类型 说明
id 字符串 任务尝试 ID
taskAttemptcounterGroup 任务尝试计数器组对象数组（JSON）/零个或多个任务尝试计数器组对象（XML） 任务尝试计数器组对象的集合

taskAttemptCounterGroup 对象的元素

项目 数据类型 说明
counterGroupName 字符串 counter 组名称
counter counter 对象数组（JSON）/零个或多个 counter 对象（XML） counter 对象集合

counter 对象元素

项目 数据类型 说明
name 字符串 counter 名称
value 长整型 计数器的值

响应示例

JSON 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0/counters

响应头

  HTTP/1.1 200 OK
  Content-Type: application/json
  Transfer-Encoding: chunked
  Server: Jetty(6.1.26)

响应正文

{
   "jobTaskAttemptCounters" : {
      "taskAttemptCounterGroup" : [
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.FileSystemCounter",
            "counter" : [
               {
                  "value" : 2363,
                  "name" : "FILE_BYTES_READ"
               },
               {
                  "value" : 54372,
                  "name" : "FILE_BYTES_WRITTEN"
               },
               {
                  "value" : 0,
                  "name" : "FILE_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "FILE_LARGE_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "FILE_WRITE_OPS"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_BYTES_READ"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_BYTES_WRITTEN"
               },
              {
                  "value" : 0,
                  "name" : "HDFS_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_LARGE_READ_OPS"
               },
               {
                  "value" : 0,
                  "name" : "HDFS_WRITE_OPS"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.TaskCounter",
            "counter" : [
               {
                  "value" : 0,
                  "name" : "COMBINE_INPUT_RECORDS"
               },
               {
                  "value" : 0,
                  "name" : "COMBINE_OUTPUT_RECORDS"
               },
               {
                  "value" : 460,
                  "name" : "REDUCE_INPUT_GROUPS"
               },
               {
                  "value" : 2235,
                  "name" : "REDUCE_SHUFFLE_BYTES"
               },
               {
                  "value" : 460,
                  "name" : "REDUCE_INPUT_RECORDS"
               },
               {
                  "value" : 0,
                  "name" : "REDUCE_OUTPUT_RECORDS"
               },
               {
                  "value" : 0,
                  "name" : "SPILLED_RECORDS"
               },
               {
                  "value" : 1,
                  "name" : "SHUFFLED_MAPS"
               },
               {
                  "value" : 0,
                  "name" : "FAILED_SHUFFLE"
               },
               {
                  "value" : 1,
                  "name" : "MERGED_MAP_OUTPUTS"
               },
               {
                  "value" : 26,
                  "name" : "GC_TIME_MILLIS"
               },
               {
                  "value" : 860,
                  "name" : "CPU_MILLISECONDS"
               },
               {
                  "value" : 107839488,
                  "name" : "PHYSICAL_MEMORY_BYTES"
               },
               {
                  "value" : 1123147776,
                  "name" : "VIRTUAL_MEMORY_BYTES"
               },
               {
                  "value" : 57475072,
                  "name" : "COMMITTED_HEAP_BYTES"
               }
            ]
         },
         {
            "counterGroupName" : "Shuffle Errors",
            "counter" : [
               {
                  "value" : 0,
                  "name" : "BAD_ID"
               },
               {
                  "value" : 0,
                  "name" : "CONNECTION"
               },
               {
                  "value" : 0,
                  "name" : "IO_ERROR"
               },
               {
                  "value" : 0,
                  "name" : "WRONG_LENGTH"
               },
               {
                  "value" : 0,
                  "name" : "WRONG_MAP"
               },
               {
                  "value" : 0,
                  "name" : "WRONG_REDUCE"
               }
            ]
         },
         {
            "counterGroupName" : "org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter",
            "counter" : [
               {
                  "value" : 0,
                  "name" : "BYTES_WRITTEN"
               }
            ]
         }
      ],
      "id" : "attempt_1326232085508_4_4_r_0_0"
   }
}

XML 响应

HTTP 请求

  GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0/counters
  Accept: application/xml

响应头

  HTTP/1.1 200 OK
  Content-Type: application/xml
  Content-Length: 2735
  Server: Jetty(6.1.26)

响应正文

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptCounters>
  <id>attempt_1326232085508_4_4_r_0_0</id>
  <taskAttemptCounterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.FileSystemCounter</counterGroupName>
    <counter>
      <name>FILE_BYTES_READ</name>
      <value>2363</value>
    </counter>
    <counter>
      <name>FILE_BYTES_WRITTEN</name>
      <value>54372</value>
    </counter>
    <counter>
      <name>FILE_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>FILE_LARGE_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>FILE_WRITE_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_BYTES_READ</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_BYTES_WRITTEN</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_LARGE_READ_OPS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>HDFS_WRITE_OPS</name>
      <value>0</value>
    </counter>
  </taskAttemptCounterGroup>
  <taskAttemptCounterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.TaskCounter</counterGroupName>
    <counter>
      <name>COMBINE_INPUT_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>COMBINE_OUTPUT_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>REDUCE_INPUT_GROUPS</name>
      <value>460</value>
    </counter>
    <counter>
      <name>REDUCE_SHUFFLE_BYTES</name>
      <value>2235</value>
    </counter>
    <counter>
      <name>REDUCE_INPUT_RECORDS</name>
      <value>460</value>
    </counter>
    <counter>
      <name>REDUCE_OUTPUT_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>SPILLED_RECORDS</name>
      <value>0</value>
    </counter>
    <counter>
      <name>SHUFFLED_MAPS</name>
      <value>1</value>
    </counter>
    <counter>
      <name>FAILED_SHUFFLE</name>
      <value>0</value>
    </counter>
    <counter>
      <name>MERGED_MAP_OUTPUTS</name>
      <value>1</value>
    </counter>
    <counter>
      <name>GC_TIME_MILLIS</name>
      <value>26</value>
    </counter>
    <counter>
      <name>CPU_MILLISECONDS</name>
      <value>860</value>
    </counter>
    <counter>
      <name>PHYSICAL_MEMORY_BYTES</name>
      <value>107839488</value>
    </counter>
    <counter>
      <name>VIRTUAL_MEMORY_BYTES</name>
      <value>1123147776</value>
    </counter>
    <counter>
      <name>COMMITTED_HEAP_BYTES</name>
      <value>57475072</value>
    </counter>
  </taskAttemptCounterGroup>
  <taskAttemptCounterGroup>
    <counterGroupName>Shuffle Errors</counterGroupName>
    <counter>
      <name>BAD_ID</name>
      <value>0</value>
    </counter>
    <counter>
      <name>CONNECTION</name>
      <value>0</value>
    </counter>
    <counter>
      <name>IO_ERROR</name>
      <value>0</value>
    </counter>
    <counter>
      <name>WRONG_LENGTH</name>
      <value>0</value>
    </counter>
    <counter>
      <name>WRONG_MAP</name>
      <value>0</value>
    </counter>
    <counter>
      <name>WRONG_REDUCE</name>
      <value>0</value>
    </counter>
  </taskAttemptCounterGroup>
  <taskAttemptCounterGroup>
    <counterGroupName>org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter</counterGroupName>
    <counter>
      <name>BYTES_WRITTEN</name>
      <value>0</value>
    </counter>
  </taskAttemptCounterGroup>
</jobTaskAttemptCounters>