spark SupportsDynamicOverwrite 源码

  • 2022-10-20
  • 浏览 (205)

spark SupportsDynamicOverwrite 代码

文件路径:/sql/catalyst/src/main/java/org/apache/spark/sql/connector/write/SupportsDynamicOverwrite.java

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.spark.sql.connector.write;

import org.apache.spark.annotation.Evolving;

/**
 * Write builder trait for tables that support dynamic partition overwrite.
 * <p>
 * A write that dynamically overwrites partitions removes all existing data in each logical
 * partition for which the write will commit new data. Any existing logical partition for which the
 * write does not contain data will remain unchanged.
 * <p>
 * This is provided to implement SQL compatible with Hive table operations but is not recommended.
 * Instead, use the {@link SupportsOverwriteV2 overwrite by filter API} to explicitly replace data.
 *
 * @since 3.0.0
 */
@Evolving
public interface SupportsDynamicOverwrite extends WriteBuilder {
  /**
   * Configures a write to dynamically replace partitions with data committed in the write.
   *
   * @return this write builder for method chaining
   */
  WriteBuilder overwriteDynamicPartitions();
}

相关信息

spark 源码目录

相关文章

spark BatchWrite 源码

spark DataWriter 源码

spark DataWriterFactory 源码

spark DeltaBatchWrite 源码

spark DeltaWrite 源码

spark DeltaWriteBuilder 源码

spark DeltaWriter 源码

spark DeltaWriterFactory 源码

spark LogicalWriteInfo 源码

spark PhysicalWriteInfo 源码

0  赞